[PATCH] hostapd: keep HE capability after channel switch in AP+STA/Mesh
The auto-ht option already kept HT and VHT support, but wasn't updated
to support HE (11ax).
Signed-off-by: Arnout Vandecappelle (Essensium/Mind)
---
This is a drive-by contribution that I just noticed while looking at the
patch. I haven't even build tested.
---
.../network/services/hostapd/patches/370-ap_sta_support.patch | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/package/network/services/hostapd/patches/370-ap_sta_support.patch
b/package/network/services/hostapd/patches/370-ap_sta_support.patch
index c5cad3bb8d..535164d802 100644
--- a/package/network/services/hostapd/patches/370-ap_sta_support.patch
+++ b/package/network/services/hostapd/patches/370-ap_sta_support.patch
@@ -235,13 +235,14 @@
--- a/hostapd/ctrl_iface.c
+++ b/hostapd/ctrl_iface.c
-@@ -2883,6 +2883,11 @@ static int hostapd_ctrl_iface_chan_switc
+@@ -2883,6 +2883,12 @@ static int hostapd_ctrl_iface_chan_switc
return 0;
}
+ if (os_strstr(pos, " auto-ht")) {
+ settings.freq_params.ht_enabled = iface->conf->ieee80211n;
+ settings.freq_params.vht_enabled = iface->conf->ieee80211ac;
++ settings.freq_params.he_enabled = iface->conf->ieee80211ax;
+ }
+
for (i = 0; i < iface->num_bss; i++) {
--
2.31.1
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Upstreaming hostapd patches 300-noscan.patch and 360-ctrl_iface_reload.patch
Hi Felix, Since I want to use these features outside of OpenWrt, I'd like to upstream the patches 300-noscan.patch and 360-ctrl_iface_reload.patch to hostap. First of all, I'd like confirmation from you that it's OK to redistribute those patches under hostap's BSD-3-Clause license rather than OpenWrt's GPL-2.0. But also, it would help if you remember if there have been any previous upstreaming attempts. I can of course search for such discussions, but I don't want to waste time on a wild goose chase. Thanks! Regards, Arnout ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH v2] ubusd: convert tx_queue to linked list
Hi Felix, On 25/03/2021 23:56, Felix Fietkau wrote: [snip] > The data structure changes that I pointed out are not a blocker for me. > I do want to see at least a per-client message limit (in order to not > regress on accidental DoS). I posted a v3 with exactly that at around the time you sent this mail. > A global one would be nice too, but is not a must-have. That one wouldn't be too hard to add, it's just that I'm not entirely sure about global variable access in the ubusd code. Is multithreading possible there? > I do agree with your proposal regarding lookup method rework, > and wire format changes are acceptable too, as long as the libubus API > doesn't change. I also took a look at that, but it was just a bit too daunting for me. Code wise it can't be too hard, but I don't understand enough about the wire format and which parts in the code it affects to have confidence in not breaking something somewhere else. Regards, Arnout > > The wire format change should probably be something generic like adding > support for having a single ubus message contain an array of data blobs > (and having libubus call data_cb() for each element of that array). > That way its use is not limited to lookups, it could also be used by > other ubus clients for sending multiple messages with less overhead. > The changes for this could probably be very simple without any major > refactoring. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH v3 1/2] ubusd: convert tx_queue to linked list
ubusd maintains a per-client tx_queue containing references to message
buffers that have not been sent yet (due to the socket blocking). This
is a fixed-size, 64-element queue.
When more than 64 elements are queued, subsequent elements are simply
dropped. Thus, a client that is waiting for those messages will block
indefinitely. In particular, this happens when more than +- 250 objects
are registered on the bus and either "ubus list" or "ubus wait_for" is
called. The responses to these requests consist of a message buffer per
object. Since in practice, ubusd will not yield between the sends of
these message buffers, the client has no time to process them and
eventually the output socket blocks. After 64 more objects, the rest is
dropped, including the final message that indicates termination. Thus,
the client waits indefinitely for the termination message.
To solve this, turn the tx_queue into a variable-sized linked list
instead of a fixed-size queue.
To maintain the linked list, an additional structure ubus_msg_buf_list
is created. It is not possible to add the linked list to ubus_msg_buf,
because that is shared between clients.
Note that this infinite tx_queue opens the door to a DoS attack. You can
open a client and a server connection, then send messages from the
client to the server without ever reading anything on the server side.
This will eventually lead to an out-of-memory. However, such a DoS
already existed anyway, it just requires opening multiple server
connections and filling up the fixed-size queue on each one. To protect
against such DoS attacks, we'd need to:
- keep a global maximum queue size that applies to all rx and tx queues
together;
- stop reading from any connection when the maximum is reached;
- close any connection when it hasn't become writeable after some
timeout.
Fixes: https://bugs.openwrt.org/index.php?do=details_id=1525
Signed-off-by: Arnout Vandecappelle (Essensium/Mind)
---
v3:
- remove the txq_ofs change
- reduce the diff a bit by adding "ub = ubl->msg;"
v2: workarounds for clang static analysis false positives:
- use list_for_each_safe instead of a while loop;
- hide the free() by moving it to ubusd.c.
---
ubusd.c | 20
ubusd.h | 11 ---
ubusd_main.c | 33 +
ubusd_proto.c | 1 +
4 files changed, 34 insertions(+), 31 deletions(-)
diff --git a/ubusd.c b/ubusd.c
index 5993653..c324c70 100644
--- a/ubusd.c
+++ b/ubusd.c
@@ -133,13 +133,25 @@ ssize_t ubus_msg_writev(int fd, struct ubus_msg_buf *ub,
size_t offset)
return ret;
}
+void ubus_msg_list_free(struct ubus_msg_buf_list *ubl)
+{
+ list_del_init(>list);
+ ubus_msg_free(ubl->msg);
+ free(ubl);
+}
+
static void ubus_msg_enqueue(struct ubus_client *cl, struct ubus_msg_buf *ub)
{
- if (cl->tx_queue[cl->txq_tail])
+ struct ubus_msg_buf_list *ubl;
+
+ ubl = calloc(1, sizeof(struct ubus_msg_buf_list));
+ if (!ubl)
return;
- cl->tx_queue[cl->txq_tail] = ubus_msg_ref(ub);
- cl->txq_tail = (cl->txq_tail + 1) % ARRAY_SIZE(cl->tx_queue);
+ INIT_LIST_HEAD(>list);
+ ubl->msg = ubus_msg_ref(ub);
+
+ list_add_tail(>tx_queue, >list);
}
/* takes the msgbuf reference */
@@ -150,7 +162,7 @@ void ubus_msg_send(struct ubus_client *cl, struct
ubus_msg_buf *ub)
if (ub->hdr.type != UBUS_MSG_MONITOR)
ubusd_monitor_message(cl, ub, true);
- if (!cl->tx_queue[cl->txq_cur]) {
+ if (list_empty(>tx_queue)) {
written = ubus_msg_writev(cl->sock.fd, ub, 0);
if (written < 0)
diff --git a/ubusd.h b/ubusd.h
index 923e43d..f34cba1 100644
--- a/ubusd.h
+++ b/ubusd.h
@@ -23,7 +23,6 @@
#include "ubusmsg.h"
#include "ubusd_acl.h"
-#define UBUSD_CLIENT_BACKLOG 32
#define UBUS_OBJ_HASH_BITS 4
extern struct blob_buf b;
@@ -36,6 +35,11 @@ struct ubus_msg_buf {
int len;
};
+struct ubus_msg_buf_list {
+ struct list_head list;
+ struct ubus_msg_buf *msg;
+};
+
struct ubus_client {
struct ubus_id id;
struct uloop_fd sock;
@@ -48,8 +52,8 @@ struct ubus_client {
struct list_head objects;
- struct ubus_msg_buf *tx_queue[UBUSD_CLIENT_BACKLOG];
- unsigned int txq_cur, txq_tail, txq_ofs;
+ struct list_head tx_queue;
+ unsigned int txq_ofs;
struct ubus_msg_buf *pending_msg;
struct ubus_msg_buf *retmsg;
@@ -72,6 +76,7 @@ struct ubus_msg_buf *ubus_msg_new(void *data, int len, bool
shared);
void ubus_msg_send(struct ubus_client *cl, struct ubus_msg_buf *ub);
ssize_t ubus_msg_writev(int fd, struct ubus_msg_buf *ub, size_t offset);
void ubus_msg_free(struct ubus_msg_buf *ub);
+void ubus_msg_list_free(struct ubus_msg_buf_list *ubl);
struct blob_attr **ubus_parse_msg(struct blob_attr *msg, size
[PATCH v3 2/2] ubusd: add per-client tx queue limit
No new message can be enqueued if this brings the total queue length of
that client over UBUS_CLIENT_MAX_TXQ_LEN.
Set UBUS_CLIENT_MAX_TXQ_LEN to UBUS_MAX_MSGLEN, i.e. 1MB. This limit
should be plenty for any practical use case.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind)
---
v3: new patch
I've tested this patch by creating about 40K objects on the bus, which
should be more than enough to reach the 1MB limit. And indeed, if I do
"ubus list | wc", I get 40K lines, but if I do "ubus list | less", it
stops at around 1000 lines (1K per object, is that realistic? Feels a
bit high to me...).
---
ubusd.c | 5 +
ubusd.h | 2 ++
ubusd_main.c | 1 +
3 files changed, 8 insertions(+)
diff --git a/ubusd.c b/ubusd.c
index c324c70..0e1b0c9 100644
--- a/ubusd.c
+++ b/ubusd.c
@@ -144,6 +144,9 @@ static void ubus_msg_enqueue(struct ubus_client *cl, struct
ubus_msg_buf *ub)
{
struct ubus_msg_buf_list *ubl;
+ if (cl->txq_len + ub->len > UBUS_CLIENT_MAX_TXQ_LEN)
+ return;
+
ubl = calloc(1, sizeof(struct ubus_msg_buf_list));
if (!ubl)
return;
@@ -152,6 +155,7 @@ static void ubus_msg_enqueue(struct ubus_client *cl, struct
ubus_msg_buf *ub)
ubl->msg = ubus_msg_ref(ub);
list_add_tail(>tx_queue, >list);
+ cl->txq_len += ub->len;
}
/* takes the msgbuf reference */
@@ -172,6 +176,7 @@ void ubus_msg_send(struct ubus_client *cl, struct
ubus_msg_buf *ub)
return;
cl->txq_ofs = written;
+ cl->txq_len = -written;
/* get an event once we can write to the socket again */
uloop_fd_add(>sock, ULOOP_READ | ULOOP_WRITE |
ULOOP_EDGE_TRIGGER);
diff --git a/ubusd.h b/ubusd.h
index f34cba1..c5d6d2a 100644
--- a/ubusd.h
+++ b/ubusd.h
@@ -24,6 +24,7 @@
#include "ubusd_acl.h"
#define UBUS_OBJ_HASH_BITS 4
+#define UBUS_CLIENT_MAX_TXQ_LENUBUS_MAX_MSGLEN
extern struct blob_buf b;
@@ -54,6 +55,7 @@ struct ubus_client {
struct list_head tx_queue;
unsigned int txq_ofs;
+ unsigned int txq_len;
struct ubus_msg_buf *pending_msg;
struct ubus_msg_buf *retmsg;
diff --git a/ubusd_main.c b/ubusd_main.c
index 3728a42..d298b51 100644
--- a/ubusd_main.c
+++ b/ubusd_main.c
@@ -74,6 +74,7 @@ static void client_cb(struct uloop_fd *sock, unsigned int
events)
}
cl->txq_ofs += written;
+ cl->txq_len -= written;
if (cl->txq_ofs < ub->len + sizeof(ub->hdr))
break;
--
2.30.2
___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] ubusd: convert tx_queue to linked list
On 25/03/2021 10:09, Sergey Zakharchenko wrote: > Hi Arnout, > > JFYI: Much of your patch resembles > https://patchwork.ozlabs.org/project/lede/patch/20180503100622.11168-1-i@qbox.audio/ > which was marked superseded for no apparent reason, but "anonymous > remembers":). It's delegated to John, so maybe he intended to rework and apply it but never got around to it. > As for the limits, one of them could include a timeout (e.g. no > progress in writing to client for over 10s => ubusd closes > connection). Oh yes, just what we need, more complexity :-) I agree though that timeout on Tx would make sense. It doesn't help for DoS attacks, but it does help to protect against a frozen client. Regards, Arnout ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH v2] ubusd: convert tx_queue to linked list
On 25/03/2021 10:48, Felix Fietkau wrote:
>
> On 2021-03-24 13:27, Arnout Vandecappelle (Essensium/Mind) wrote:
>> ubusd maintains a per-client tx_queue containing references to message
>> buffers that have not been sent yet (due to the socket blocking). This
>> is a fixed-size, 64-element queue.
>>
>> When more than 64 elements are queued, subsequent elements are simply
>> dropped. Thus, a client that is waiting for those messages will block
>> indefinitely. In particular, this happens when more than +- 250 objects
>> are registered on the bus and either "ubus list" or "ubus wait_for" is
>> called. The responses to these requests consist of a message buffer per
>> object. Since in practice, ubusd will not yield between the sends of
>> these message buffers, the client has no time to process them and
>> eventually the output socket blocks. After 64 more objects, the rest is
>> dropped, including the final message that indicates termination. Thus,
>> the client waits indefinitely for the termination message.
>>
>> To solve this, turn the tx_queue into a variable-sized linked list
>> instead of a fixed-size queue.
> In order to reduce memory allocation overhead, I would propose the
> following:
>
> struct ubus_msg_backlog {
> struct ubus_msg_backlog *next;
> struct ubus_msg_buf *msg[UBUSD_CLIENT_BACKLOG];
> int tail;
> };
This saves space by open-coding a single-linked list rather than using the
normal double-linked list. This comes at the cost of iterating over the entire
list in order to append to it.
It also saves space by dropping the offset member. But for that, we don't need
to go to this extra structure.
Applying those to "optimisations" to struct ubus_msg_buf_list would make that
one 8 bytes (compared to 16 now). So that's 8 bytes per queued buffer, in
addition to the 24 bytes of struct ubus_msg_buf and the actual buffer itself,
which you can expect to be hundreds of bytes.
struct ubus_msg_backlog is 24 bytes (assuming UBUSD_CLIENT_BACKLOG is 4),
that's 6 bytes per element. So not much gained here. Increasing
UBUSD_CLIENT_BACKLOG will decrease the overhead for large backlog, but gives a
much bigger overhead for smaller backlog. So not a clear win.
Finally, the backlog should normally be empty. It's pretty unusual for a ubus
message to take more then one write to transfer.
In conclusion:
- I think your proposal doesn't save much;
- and it complicates things quite a bit;
- in addition, the single-linked list potentially poses significant time
overhead.
If you want to save a few bytes, it does make sense to move the offset back to
struct ubus_client.
If you *really* want to save space, and time as well, it would be better to
optimise ubusd_handle_lookup. That currently sends a separate, relatively small,
message for every object. The overhead of the struct ubus_msg_buf dwarfs the
overhead of struct ubus_msg_buf_list by quite a lot, and in addition there's
overhead on the wire as well. It shouldn't be too hard to change ubus_lookup_cb
to handle a list rather than a single object.
Maybe I should have gone down that path. I hadn't thought of it at the time.
>
> To struct ubus_client add these:
>
> struct ubus_msg_backlog *backlog;
> int backlog_head;
>
> After sending messages from tx_queue, you pull in messages from
> cl->backlog, incrementing backlog_head.
> Once cl->backlog_head == backlog->tail, you set cl->backlog to
> backlog->next and free backlog.
>
>> To maintain the linked list, an additional structure ubus_msg_buf_list
>> is created. We could also have added the linked list to the ubus_msg_buf
>> struct itself, but it is not relevant in the many other uses of the
>> ubus_msg_buf struct so it would just complicate things.
> Adding the linked list to ubus_msg_buf doesn't work, because a single
> message can be queued for multiple receivers. This mistake was already
> made by previous attempts at introducing a linked list for messages.
Right, ubus_msg_free doesn't actually free it, it decreases the refcount. I
forgot that. Good thing I didn't implement it that way then :-)
>> The txq_off variable was used to keep track of which part of the current
>> message was already written, in case only a partial write was possible.
>> We take this opportunity to also move that variable under the
>> ubus_msg_buf_list structure (and rename it to "offset", and change its
>> type to size_t). This makes it clearer that it is related to that
>> particular queued message and not to the queue as a whole.
>>
>> Note that this infinite tx_queue opens the door to a DoS attack. You can
>> open a client and a server connection, then send messages from the
Re: [PATCH] ubusd: convert tx_queue to linked list
On 24/03/2021 09:31, Petr Štetiar wrote: > Arnout Vandecappelle (Essensium/Mind) [2021-03-23 16:23:25]: > > Hi, > >> Note that this infinite tx_queue opens the door to a DoS attack. > > memory is not infinite, so I think, that some sensible upper limit should be > kept. It might be much higher by default since it's dynamic now, but still > should be capped and lifted if needed. I agree, however: - the limit should be a global one (not per client); - it should take into account Rx, Tx and monitor buffers; - it's not exactly simple to implement (or I don't see a simple way to implement it); - I don't have time to work on this; - it's anyway a rather theoretical issue which already exists anyway, as explained in the commit message; - the fix is fixing an actual use case that is not theoretical at all (as in, we have a device that doesn't boot due to this issue). Regards, Arnout ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH v2] ubusd: convert tx_queue to linked list
ubusd maintains a per-client tx_queue containing references to message
buffers that have not been sent yet (due to the socket blocking). This
is a fixed-size, 64-element queue.
When more than 64 elements are queued, subsequent elements are simply
dropped. Thus, a client that is waiting for those messages will block
indefinitely. In particular, this happens when more than +- 250 objects
are registered on the bus and either "ubus list" or "ubus wait_for" is
called. The responses to these requests consist of a message buffer per
object. Since in practice, ubusd will not yield between the sends of
these message buffers, the client has no time to process them and
eventually the output socket blocks. After 64 more objects, the rest is
dropped, including the final message that indicates termination. Thus,
the client waits indefinitely for the termination message.
To solve this, turn the tx_queue into a variable-sized linked list
instead of a fixed-size queue.
To maintain the linked list, an additional structure ubus_msg_buf_list
is created. We could also have added the linked list to the ubus_msg_buf
struct itself, but it is not relevant in the many other uses of the
ubus_msg_buf struct so it would just complicate things.
The txq_off variable was used to keep track of which part of the current
message was already written, in case only a partial write was possible.
We take this opportunity to also move that variable under the
ubus_msg_buf_list structure (and rename it to "offset", and change its
type to size_t). This makes it clearer that it is related to that
particular queued message and not to the queue as a whole.
Note that this infinite tx_queue opens the door to a DoS attack. You can
open a client and a server connection, then send messages from the
client to the server without ever reading anything on the server side.
This will eventually lead to an out-of-memory. However, such a DoS
already existed anyway, it just requires opening multiple server
connections and filling up the fixed-size queue on each one. To protect
against such DoS attacks, we'd need to:
- keep a global maximum queue size that applies to all rx and tx queues
together;
- stop reading from any connection when the maximum is reached;
- close any connection when it hasn't become writeable after some
timeout.
Fixes: https://bugs.openwrt.org/index.php?do=details_id=1525
Signed-off-by: Arnout Vandecappelle (Essensium/Mind)
---
v2: workarounds for clang static analysis false positives:
- use list_for_each_safe instead of a while loop;
- hide the free() by moving it to ubusd.c.
---
ubusd.c | 30 +-
ubusd.h | 11 ---
ubusd_main.c | 38 +++---
ubusd_proto.c | 1 +
4 files changed, 41 insertions(+), 39 deletions(-)
diff --git a/ubusd.c b/ubusd.c
index 5993653..f8e33f8 100644
--- a/ubusd.c
+++ b/ubusd.c
@@ -133,24 +133,38 @@ ssize_t ubus_msg_writev(int fd, struct ubus_msg_buf *ub,
size_t offset)
return ret;
}
-static void ubus_msg_enqueue(struct ubus_client *cl, struct ubus_msg_buf *ub)
+void ubus_msg_list_free(struct ubus_msg_buf_list *ubl)
{
- if (cl->tx_queue[cl->txq_tail])
+ list_del_init(>list);
+ ubus_msg_free(ubl->msg);
+ free(ubl);
+}
+
+static void ubus_msg_enqueue(struct ubus_client *cl, struct ubus_msg_buf *ub,
+size_t offset)
+{
+ struct ubus_msg_buf_list *ubl;
+
+ ubl = calloc(1, sizeof(struct ubus_msg_buf_list));
+ if (!ubl)
return;
- cl->tx_queue[cl->txq_tail] = ubus_msg_ref(ub);
- cl->txq_tail = (cl->txq_tail + 1) % ARRAY_SIZE(cl->tx_queue);
+ INIT_LIST_HEAD(>list);
+ ubl->msg = ubus_msg_ref(ub);
+ ubl->offset = offset;
+
+ list_add_tail(>tx_queue, >list);
}
/* takes the msgbuf reference */
void ubus_msg_send(struct ubus_client *cl, struct ubus_msg_buf *ub)
{
- ssize_t written;
+ ssize_t written = 0;
if (ub->hdr.type != UBUS_MSG_MONITOR)
ubusd_monitor_message(cl, ub, true);
- if (!cl->tx_queue[cl->txq_cur]) {
+ if (list_empty(>tx_queue)) {
written = ubus_msg_writev(cl->sock.fd, ub, 0);
if (written < 0)
@@ -159,10 +173,8 @@ void ubus_msg_send(struct ubus_client *cl, struct
ubus_msg_buf *ub)
if (written >= (ssize_t) (ub->len + sizeof(ub->hdr)))
return;
- cl->txq_ofs = written;
-
/* get an event once we can write to the socket again */
uloop_fd_add(>sock, ULOOP_READ | ULOOP_WRITE |
ULOOP_EDGE_TRIGGER);
}
- ubus_msg_enqueue(cl, ub);
+ ubus_msg_enqueue(cl, ub, written);
}
diff --git a/ubusd.h b/ubusd.h
index 923e43d..e20e55a 100644
--- a/ubusd.h
+++ b/ubusd.h
@@ -23,7 +23,6 @@
#include "ubusmsg.h&qu
Re: [PATCH] ubusd: convert tx_queue to linked list
On 24/03/2021 08:52, Arnout Vandecappelle wrote: > > > On 23/03/2021 17:16, Petr Štetiar wrote: >> Arnout Vandecappelle (Essensium/Mind) [2021-03-23 16:23:25]: >> >> Hi, >> >>> To maintain the linked list, an additional structure ubus_msg_buf_list >>> is created. We could also have added the linked list to the ubus_msg_buf >>> struct itself, but it is not relevant in the many other uses of the >>> ubus_msg_buf struct so it would just complicate things. >> >> I've just tried to run CI pipeline[1] on this patch and got following >> complaints >> from clang static analyzer[2]: >> >> ubusd_main.c:33:3: warning: Use of memory after it is freed [unix.Malloc] >> ubus_msg_list_free(ubl); >> ^~~ >> >> ubusd_main.c:76:39: warning: Use of memory after it is freed [unix.Malloc] >> written = ubus_msg_writev(sock->fd, ubl->msg, ubl->offset); >> ^~~~ >> >> 1. https://gitlab.com/ynezz/openwrt-ubus/-/pipelines/275104805 >> 2. >> https://ynezz.gitlab.io/-/openwrt-ubus/-/jobs/1121145462/artifacts/build/scan/2021-03-23-154521-70-1/index.html > > This is a false positive. I suspect that clang doesn't see that !list_empty() > implies that the list is not empty. I'll try to rewrite using assigning to ubl > in the while loop header rather than in the body. Duh, that doesn't work - list_first_entry doesn't give NULL if the list is empty... I'm not sure how to fix this. Any ideas? Regards, Arnout ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: [PATCH] ubusd: convert tx_queue to linked list
On 23/03/2021 17:16, Petr Štetiar wrote: > Arnout Vandecappelle (Essensium/Mind) [2021-03-23 16:23:25]: > > Hi, > >> To maintain the linked list, an additional structure ubus_msg_buf_list >> is created. We could also have added the linked list to the ubus_msg_buf >> struct itself, but it is not relevant in the many other uses of the >> ubus_msg_buf struct so it would just complicate things. > > I've just tried to run CI pipeline[1] on this patch and got following > complaints > from clang static analyzer[2]: > > ubusd_main.c:33:3: warning: Use of memory after it is freed [unix.Malloc] > ubus_msg_list_free(ubl); > ^~~ > > ubusd_main.c:76:39: warning: Use of memory after it is freed [unix.Malloc] > written = ubus_msg_writev(sock->fd, ubl->msg, ubl->offset); > ^~~~ > > 1. https://gitlab.com/ynezz/openwrt-ubus/-/pipelines/275104805 > 2. > https://ynezz.gitlab.io/-/openwrt-ubus/-/jobs/1121145462/artifacts/build/scan/2021-03-23-154521-70-1/index.html This is a false positive. I suspect that clang doesn't see that !list_empty() implies that the list is not empty. I'll try to rewrite using assigning to ubl in the while loop header rather than in the body. Regards, Arnout ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
[PATCH] ubusd: convert tx_queue to linked list
ubusd maintains a per-client tx_queue containing references to message
buffers that have not been sent yet (due to the socket blocking). This
is a fixed-size, 64-element queue.
When more than 64 elements are queued, subsequent elements are simply
dropped. Thus, a client that is waiting for those messages will block
indefinitely. In particular, this happens when more than +- 250 objects
are registered on the bus and either "ubus list" or "ubus wait_for" is
called. The responses to these requests consist of a message buffer per
object. Since in practice, ubusd will not yield between the sends of
these message buffers, the client has no time to process them and
eventually the output socket blocks. After 64 more objects, the rest is
dropped, including the final message that indicates termination. Thus,
the client waits indefinitely for the termination message.
To solve this, turn the tx_queue into a variable-sized linked list
instead of a fixed-size queue.
To maintain the linked list, an additional structure ubus_msg_buf_list
is created. We could also have added the linked list to the ubus_msg_buf
struct itself, but it is not relevant in the many other uses of the
ubus_msg_buf struct so it would just complicate things.
The txq_off variable was used to keep track of which part of the current
message was already written, in case only a partial write was possible.
We take this opportunity to also move that variable under the
ubus_msg_buf_list structure (and rename it to "offset", and change its
type to size_t). This makes it clearer that it is related to that
particular queued message and not to the queue as a whole.
Note that this infinite tx_queue opens the door to a DoS attack. You can
open a client and a server connection, then send messages from the
client to the server without ever reading anything on the server side.
This will eventually lead to an out-of-memory. However, such a DoS
already existed anyway, it just requires opening multiple server
connections and filling up the fixed-size queue on each one. To protect
against such DoS attacks, we'd need to:
- keep a global maximum queue size that applies to all rx and tx queues
together;
- stop reading from any connection when the maximum is reached;
- close any connection when it hasn't become writeable after some
timeout.
Fixes: https://bugs.openwrt.org/index.php?do=details_id=1525
Signed-off-by: Arnout Vandecappelle (Essensium/Mind)
---
ubusd.c | 23 ++-
ubusd.h | 10 +++---
ubusd_main.c | 45 +
ubusd_proto.c | 1 +
4 files changed, 43 insertions(+), 36 deletions(-)
diff --git a/ubusd.c b/ubusd.c
index 5993653..b7cafaf 100644
--- a/ubusd.c
+++ b/ubusd.c
@@ -133,24 +133,31 @@ ssize_t ubus_msg_writev(int fd, struct ubus_msg_buf *ub,
size_t offset)
return ret;
}
-static void ubus_msg_enqueue(struct ubus_client *cl, struct ubus_msg_buf *ub)
+static void ubus_msg_enqueue(struct ubus_client *cl, struct ubus_msg_buf *ub,
+size_t offset)
{
- if (cl->tx_queue[cl->txq_tail])
+ struct ubus_msg_buf_list *ubl;
+
+ ubl = calloc(1, sizeof(struct ubus_msg_buf_list));
+ if (!ubl)
return;
- cl->tx_queue[cl->txq_tail] = ubus_msg_ref(ub);
- cl->txq_tail = (cl->txq_tail + 1) % ARRAY_SIZE(cl->tx_queue);
+ INIT_LIST_HEAD(>list);
+ ubl->msg = ubus_msg_ref(ub);
+ ubl->offset = offset;
+
+ list_add_tail(>tx_queue, >list);
}
/* takes the msgbuf reference */
void ubus_msg_send(struct ubus_client *cl, struct ubus_msg_buf *ub)
{
- ssize_t written;
+ ssize_t written = 0;
if (ub->hdr.type != UBUS_MSG_MONITOR)
ubusd_monitor_message(cl, ub, true);
- if (!cl->tx_queue[cl->txq_cur]) {
+ if (list_empty(>tx_queue)) {
written = ubus_msg_writev(cl->sock.fd, ub, 0);
if (written < 0)
@@ -159,10 +166,8 @@ void ubus_msg_send(struct ubus_client *cl, struct
ubus_msg_buf *ub)
if (written >= (ssize_t) (ub->len + sizeof(ub->hdr)))
return;
- cl->txq_ofs = written;
-
/* get an event once we can write to the socket again */
uloop_fd_add(>sock, ULOOP_READ | ULOOP_WRITE |
ULOOP_EDGE_TRIGGER);
}
- ubus_msg_enqueue(cl, ub);
+ ubus_msg_enqueue(cl, ub, written);
}
diff --git a/ubusd.h b/ubusd.h
index 923e43d..4131274 100644
--- a/ubusd.h
+++ b/ubusd.h
@@ -23,7 +23,6 @@
#include "ubusmsg.h"
#include "ubusd_acl.h"
-#define UBUSD_CLIENT_BACKLOG 32
#define UBUS_OBJ_HASH_BITS 4
extern struct blob_buf b;
@@ -36,6 +35,12 @@ struct ubus_msg_buf {
int len;
};
+struct ubus_msg_buf_list {
+ struct list_head list;
+ struct ubus_msg_buf *msg;
+ size_
