Re: [PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Jiri Kosina 
On Wed, 8 Mar 2017, Eric Dumazet wrote:

> > +++ b/net/sched/sch_qfq.c
> > @@ -494,6 +494,8 @@ static int qfq_change_class(struct Qdisc *sch, u32 
> > classid, u32 parentid,
> > goto destroy_class;
> > }
> >  
> > +   if (cl->qdisc != _qdisc)
> > +   qdisc_hash_add(cl->qdisc, true);
> 
> 
> Please move the test in qdisc_hash_add() instead of copy/pasting it all
> over the places ?

Well, qdisc_hash_add() has a WARN_ON() (inherited from what 
qdisc_list_add() used to do) for that particular case to catch cases where 
singleton qdisc would make it there from other places by mistake. By 
putting this test there we'll effectively giving up on this warning should 
it ever point to a bug.

Thanks,

-- 
Jiri Kosina
SUSE Labs

Re: [PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Jiri Kosina 
On Wed, 8 Mar 2017, Eric Dumazet wrote:

> > +++ b/net/sched/sch_qfq.c
> > @@ -494,6 +494,8 @@ static int qfq_change_class(struct Qdisc *sch, u32 
> > classid, u32 parentid,
> > goto destroy_class;
> > }
> >  
> > +   if (cl->qdisc != _qdisc)
> > +   qdisc_hash_add(cl->qdisc, true);
> 
> 
> Please move the test in qdisc_hash_add() instead of copy/pasting it all
> over the places ?

Well, qdisc_hash_add() has a WARN_ON() (inherited from what 
qdisc_list_add() used to do) for that particular case to catch cases where 
singleton qdisc would make it there from other places by mistake. By 
putting this test there we'll effectively giving up on this warning should 
it ever point to a bug.

Thanks,

-- 
Jiri Kosina
SUSE Labs

Re: [PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Eric Dumazet 
On Wed, 2017-03-08 at 13:03 +0100, Jiri Kosina wrote:


> +++ b/net/sched/sch_qfq.c
> @@ -494,6 +494,8 @@ static int qfq_change_class(struct Qdisc *sch, u32 
> classid, u32 parentid,
>   goto destroy_class;
>   }
>  
> + if (cl->qdisc != _qdisc)
> + qdisc_hash_add(cl->qdisc, true);


Please move the test in qdisc_hash_add() instead of copy/pasting it all
over the places ?

This is control path, keep it small, thanks !

Re: [PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Eric Dumazet 
On Wed, 2017-03-08 at 13:03 +0100, Jiri Kosina wrote:


> +++ b/net/sched/sch_qfq.c
> @@ -494,6 +494,8 @@ static int qfq_change_class(struct Qdisc *sch, u32 
> classid, u32 parentid,
>   goto destroy_class;
>   }
>  
> + if (cl->qdisc != _qdisc)
> + qdisc_hash_add(cl->qdisc, true);


Please move the test in qdisc_hash_add() instead of copy/pasting it all
over the places ?

This is control path, keep it small, thanks !

Re: [PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Jiri Pirko 
Wed, Mar 08, 2017 at 01:03:42PM CET, ji...@kernel.org wrote:
>From: Jiri Kosina 
>
>The original reason [1] for having hidden qdiscs (potential scalability
>issues in qdisc_match_from_root() with single linked list in case of large
>amount of qdiscs) has been invalidated by 59cc1f61f0 ("net: sched: convert
>qdisc linked list to hashtable").
>
>This allows us for bringing more clarity and determinism into the dump by
>making default pfifo qdiscs visible.
>
>We're not turning this on by default though, at it was deemed [2] too
>intrusive / unnecessary change of default behavior towards userspace.
>Instead, TCA_DUMP_INVISIBLE netlink attribute is introduced, which allows
>applications to request complete qdisc hierarchy dump, including the
>ones that have always been implicit/invisible.
>
>Singleton noop_qdisc stays invisible, as teaching the whole infrastructure
>about singletons would require quite some surgery with very little gain
>(seeing no qdisc or seeing noop qdisc in the dump is probably setting
>the same user expectation).
>
>[1] 
>http://lkml.kernel.org/r/1460732328.10638.74.ca...@edumazet-glaptop3.roam.corp.google.com
>[2] 
>http://lkml.kernel.org/r/20161021.105935.1907696543877061916.da...@davemloft.net
>
>Signed-off-by: Jiri Kosina 
>---

[...]


>diff --git a/include/uapi/linux/rtnetlink.h b/include/uapi/linux/rtnetlink.h
>index 262f0379d83a..c7de00e09797 100644
>--- a/include/uapi/linux/rtnetlink.h
>+++ b/include/uapi/linux/rtnetlink.h
>@@ -542,6 +542,7 @@ enum {
>   TCA_FCNT,
>   TCA_STATS2,
>   TCA_STAB,
>+  TCA_DUMP_INVISIBLE,
>   TCA_PAD,

You are changing UAPI value of TCA_PAD...


>   __TCA_MAX
> };

Re: [PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Jiri Pirko 
Wed, Mar 08, 2017 at 01:03:42PM CET, ji...@kernel.org wrote:
>From: Jiri Kosina 
>
>The original reason [1] for having hidden qdiscs (potential scalability
>issues in qdisc_match_from_root() with single linked list in case of large
>amount of qdiscs) has been invalidated by 59cc1f61f0 ("net: sched: convert
>qdisc linked list to hashtable").
>
>This allows us for bringing more clarity and determinism into the dump by
>making default pfifo qdiscs visible.
>
>We're not turning this on by default though, at it was deemed [2] too
>intrusive / unnecessary change of default behavior towards userspace.
>Instead, TCA_DUMP_INVISIBLE netlink attribute is introduced, which allows
>applications to request complete qdisc hierarchy dump, including the
>ones that have always been implicit/invisible.
>
>Singleton noop_qdisc stays invisible, as teaching the whole infrastructure
>about singletons would require quite some surgery with very little gain
>(seeing no qdisc or seeing noop qdisc in the dump is probably setting
>the same user expectation).
>
>[1] 
>http://lkml.kernel.org/r/1460732328.10638.74.ca...@edumazet-glaptop3.roam.corp.google.com
>[2] 
>http://lkml.kernel.org/r/20161021.105935.1907696543877061916.da...@davemloft.net
>
>Signed-off-by: Jiri Kosina 
>---

[...]


>diff --git a/include/uapi/linux/rtnetlink.h b/include/uapi/linux/rtnetlink.h
>index 262f0379d83a..c7de00e09797 100644
>--- a/include/uapi/linux/rtnetlink.h
>+++ b/include/uapi/linux/rtnetlink.h
>@@ -542,6 +542,7 @@ enum {
>   TCA_FCNT,
>   TCA_STATS2,
>   TCA_STAB,
>+  TCA_DUMP_INVISIBLE,
>   TCA_PAD,

You are changing UAPI value of TCA_PAD...


>   __TCA_MAX
> };

[PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Jiri Kosina 
From: Jiri Kosina 

The original reason [1] for having hidden qdiscs (potential scalability
issues in qdisc_match_from_root() with single linked list in case of large
amount of qdiscs) has been invalidated by 59cc1f61f0 ("net: sched: convert
qdisc linked list to hashtable").

This allows us for bringing more clarity and determinism into the dump by
making default pfifo qdiscs visible.

We're not turning this on by default though, at it was deemed [2] too
intrusive / unnecessary change of default behavior towards userspace.
Instead, TCA_DUMP_INVISIBLE netlink attribute is introduced, which allows
applications to request complete qdisc hierarchy dump, including the
ones that have always been implicit/invisible.

Singleton noop_qdisc stays invisible, as teaching the whole infrastructure
about singletons would require quite some surgery with very little gain
(seeing no qdisc or seeing noop qdisc in the dump is probably setting
the same user expectation).

[1] 
http://lkml.kernel.org/r/1460732328.10638.74.ca...@edumazet-glaptop3.roam.corp.google.com
[2] 
http://lkml.kernel.org/r/20161021.105935.1907696543877061916.da...@davemloft.net

Signed-off-by: Jiri Kosina 
---

v1 -> v2: introduce exception for singleton noop_qdisc

 include/net/pkt_sched.h|  2 +-
 include/net/sch_generic.h  |  1 +
 include/uapi/linux/rtnetlink.h |  1 +
 net/sched/sch_api.c| 42 ++
 net/sched/sch_cbq.c|  5 +
 net/sched/sch_drr.c|  2 ++
 net/sched/sch_dsmark.c |  2 ++
 net/sched/sch_generic.c|  2 +-
 net/sched/sch_hfsc.c   |  4 
 net/sched/sch_htb.c|  2 ++
 net/sched/sch_mq.c |  2 +-
 net/sched/sch_mqprio.c |  2 +-
 net/sched/sch_multiq.c |  2 ++
 net/sched/sch_prio.c   |  5 -
 net/sched/sch_qfq.c|  2 ++
 net/sched/sch_red.c|  2 ++
 net/sched/sch_sfb.c|  2 ++
 net/sched/sch_tbf.c|  2 ++
 18 files changed, 65 insertions(+), 17 deletions(-)

diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index cd334c9584e9..0625eac2c601 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -90,7 +90,7 @@ int unregister_qdisc(struct Qdisc_ops *qops);
 void qdisc_get_default(char *id, size_t len);
 int qdisc_set_default(const char *id);
 
-void qdisc_hash_add(struct Qdisc *q);
+void qdisc_hash_add(struct Qdisc *q, bool invisible);
 void qdisc_hash_del(struct Qdisc *q);
 struct Qdisc *qdisc_lookup(struct net_device *dev, u32 handle);
 struct Qdisc *qdisc_lookup_class(struct net_device *dev, u32 handle);
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index e6aa0a249672..e7dca250d115 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -66,6 +66,7 @@ struct Qdisc {
 #define TCQ_F_NOPARENT 0x40 /* root of its hierarchy :
  * qdisc_tree_decrease_qlen() should stop.
  */
+#define TCQ_F_INVISIBLE0x80 /* invisible by default in dump */
u32 limit;
const struct Qdisc_ops  *ops;
struct qdisc_size_table __rcu *stab;
diff --git a/include/uapi/linux/rtnetlink.h b/include/uapi/linux/rtnetlink.h
index 262f0379d83a..c7de00e09797 100644
--- a/include/uapi/linux/rtnetlink.h
+++ b/include/uapi/linux/rtnetlink.h
@@ -542,6 +542,7 @@ enum {
TCA_FCNT,
TCA_STATS2,
TCA_STAB,
+   TCA_DUMP_INVISIBLE,
TCA_PAD,
__TCA_MAX
 };
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index 206dc24add3a..8e4e6ab1847a 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -274,7 +274,7 @@ static struct Qdisc *qdisc_match_from_root(struct Qdisc 
*root, u32 handle)
return NULL;
 }
 
-void qdisc_hash_add(struct Qdisc *q)
+void qdisc_hash_add(struct Qdisc *q, bool invisible)
 {
if ((q->parent != TC_H_ROOT) && !(q->flags & TCQ_F_INGRESS)) {
struct Qdisc *root = qdisc_dev(q)->qdisc;
@@ -282,6 +282,8 @@ void qdisc_hash_add(struct Qdisc *q)
WARN_ON_ONCE(root == _qdisc);
ASSERT_RTNL();
hash_add_rcu(qdisc_dev(q)->qdisc_hash, >hash, q->handle);
+   if (invisible)
+   q->flags |= TCQ_F_INVISIBLE;
}
 }
 EXPORT_SYMBOL(qdisc_hash_add);
@@ -1004,7 +1006,7 @@ static struct Qdisc *qdisc_create(struct net_device *dev,
goto err_out4;
}
 
-   qdisc_hash_add(sch);
+   qdisc_hash_add(sch, false);
 
return sch;
}
@@ -1400,9 +1402,14 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct 
Qdisc *q, u32 clid,
return -1;
 }
 
-static bool tc_qdisc_dump_ignore(struct Qdisc *q)
+static bool tc_qdisc_dump_ignore(struct Qdisc *q, bool dump_invisible)
 {
-   return (q->flags & TCQ_F_BUILTIN) ?

[PATCH v2 1/2] net: sched: make default fifo qdiscs appear in the dump

2017-03-08 Thread Jiri Kosina 
From: Jiri Kosina 

The original reason [1] for having hidden qdiscs (potential scalability
issues in qdisc_match_from_root() with single linked list in case of large
amount of qdiscs) has been invalidated by 59cc1f61f0 ("net: sched: convert
qdisc linked list to hashtable").

This allows us for bringing more clarity and determinism into the dump by
making default pfifo qdiscs visible.

We're not turning this on by default though, at it was deemed [2] too
intrusive / unnecessary change of default behavior towards userspace.
Instead, TCA_DUMP_INVISIBLE netlink attribute is introduced, which allows
applications to request complete qdisc hierarchy dump, including the
ones that have always been implicit/invisible.

Singleton noop_qdisc stays invisible, as teaching the whole infrastructure
about singletons would require quite some surgery with very little gain
(seeing no qdisc or seeing noop qdisc in the dump is probably setting
the same user expectation).

[1] 
http://lkml.kernel.org/r/1460732328.10638.74.ca...@edumazet-glaptop3.roam.corp.google.com
[2] 
http://lkml.kernel.org/r/20161021.105935.1907696543877061916.da...@davemloft.net

Signed-off-by: Jiri Kosina 
---

v1 -> v2: introduce exception for singleton noop_qdisc

 include/net/pkt_sched.h|  2 +-
 include/net/sch_generic.h  |  1 +
 include/uapi/linux/rtnetlink.h |  1 +
 net/sched/sch_api.c| 42 ++
 net/sched/sch_cbq.c|  5 +
 net/sched/sch_drr.c|  2 ++
 net/sched/sch_dsmark.c |  2 ++
 net/sched/sch_generic.c|  2 +-
 net/sched/sch_hfsc.c   |  4 
 net/sched/sch_htb.c|  2 ++
 net/sched/sch_mq.c |  2 +-
 net/sched/sch_mqprio.c |  2 +-
 net/sched/sch_multiq.c |  2 ++
 net/sched/sch_prio.c   |  5 -
 net/sched/sch_qfq.c|  2 ++
 net/sched/sch_red.c|  2 ++
 net/sched/sch_sfb.c|  2 ++
 net/sched/sch_tbf.c|  2 ++
 18 files changed, 65 insertions(+), 17 deletions(-)

diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index cd334c9584e9..0625eac2c601 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -90,7 +90,7 @@ int unregister_qdisc(struct Qdisc_ops *qops);
 void qdisc_get_default(char *id, size_t len);
 int qdisc_set_default(const char *id);
 
-void qdisc_hash_add(struct Qdisc *q);
+void qdisc_hash_add(struct Qdisc *q, bool invisible);
 void qdisc_hash_del(struct Qdisc *q);
 struct Qdisc *qdisc_lookup(struct net_device *dev, u32 handle);
 struct Qdisc *qdisc_lookup_class(struct net_device *dev, u32 handle);
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index e6aa0a249672..e7dca250d115 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -66,6 +66,7 @@ struct Qdisc {
 #define TCQ_F_NOPARENT 0x40 /* root of its hierarchy :
  * qdisc_tree_decrease_qlen() should stop.
  */
+#define TCQ_F_INVISIBLE0x80 /* invisible by default in dump */
u32 limit;
const struct Qdisc_ops  *ops;
struct qdisc_size_table __rcu *stab;
diff --git a/include/uapi/linux/rtnetlink.h b/include/uapi/linux/rtnetlink.h
index 262f0379d83a..c7de00e09797 100644
--- a/include/uapi/linux/rtnetlink.h
+++ b/include/uapi/linux/rtnetlink.h
@@ -542,6 +542,7 @@ enum {
TCA_FCNT,
TCA_STATS2,
TCA_STAB,
+   TCA_DUMP_INVISIBLE,
TCA_PAD,
__TCA_MAX
 };
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index 206dc24add3a..8e4e6ab1847a 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -274,7 +274,7 @@ static struct Qdisc *qdisc_match_from_root(struct Qdisc 
*root, u32 handle)
return NULL;
 }
 
-void qdisc_hash_add(struct Qdisc *q)
+void qdisc_hash_add(struct Qdisc *q, bool invisible)
 {
if ((q->parent != TC_H_ROOT) && !(q->flags & TCQ_F_INGRESS)) {
struct Qdisc *root = qdisc_dev(q)->qdisc;
@@ -282,6 +282,8 @@ void qdisc_hash_add(struct Qdisc *q)
WARN_ON_ONCE(root == _qdisc);
ASSERT_RTNL();
hash_add_rcu(qdisc_dev(q)->qdisc_hash, >hash, q->handle);
+   if (invisible)
+   q->flags |= TCQ_F_INVISIBLE;
}
 }
 EXPORT_SYMBOL(qdisc_hash_add);
@@ -1004,7 +1006,7 @@ static struct Qdisc *qdisc_create(struct net_device *dev,
goto err_out4;
}
 
-   qdisc_hash_add(sch);
+   qdisc_hash_add(sch, false);
 
return sch;
}
@@ -1400,9 +1402,14 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct 
Qdisc *q, u32 clid,
return -1;
 }
 
-static bool tc_qdisc_dump_ignore(struct Qdisc *q)
+static bool tc_qdisc_dump_ignore(struct Qdisc *q, bool dump_invisible)
 {
-   return (q->flags & TCQ_F_BUILTIN) ? true : false;
+   if (q->flags