Re: Is anybody using ng_pipe?
Writing a new netgraph node is relatively simple. Take ng_sample.c and ng_sample.h and copy them. Change names to suit, and add your own code in the middle. use one of 50 other nodes as examples. No matter what you want to do one of them already does it. -- +--\ _ __ | __--_|\ Julian Elischer\ \\ U \/ / On assignment | / \ jul...@elischer.org \ \ USA\ in a very strange | ( OZ) \-->x ___ | country ! +- X_.---._/ Mountain View, California \_/ \\ v sp; \\ v ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
> On Tue, Aug 18, 2020 at 2:43 PM Eugene Grosbein wrote: > > Sorry, missed that. But why wasn't possible? > > There's a daemon running on the system that handles most network > configuration. It's quite inflexible and will override any manual > configuration changes. It manages firewall rules but is ignorant of > netgraph, so it will remove any dummynet rules but leave netgraph > configuration alone. It was significantly easier to just use ng_pipe, > even after having to fix or work around the bugs, than it was to fight > the daemon. > > On Tue, Aug 18, 2020 at 2:56 PM Marko Zec wrote: > > The probability that a frame is completely unaffected by BER events, > > and thus shouldn't be dropped, is currently computed as > > > > Ppass(BER, plen) = Psingle_bit_unaffected(BER) ^ Nbits(plen) > > The problem is in its calculation of Psingle_bit_unaffected(BER). The > BER is the fraction of bits that are affected, therefore it is the > probability that a bit is affected. But for some reason, > Psingle_bit_unaffected(BER) is calculated as 1 - 1/BER rather than 1 - > BER. This leads to the probability table being wrong. For example, > given a BER of 2350, the probability that a 1500-byte packet is > not dropped is: Is this a confusion over Bit Error Rate vis Bit Error Ratio? A BER of 2350 I must assume is 1 bit in 2350 bits. 1 - 1/BER looks correct to me for a Bit Error Rate 1 - BER looks correct to me for a Bit Error Ratio (usually a percentage) > > (1 - 2350/2**48)**(1500 * 8), which is approximately 99.00%. > > However, ng_pipe calculates a fixed-point probability value of > 281460603879001. To calculate whether a frame should be dropped, > ng_pipe takes this probability value and shifts it right by 17, > yielding 2147373991. It then calls rand() to generate a random number > in the range [0,2**31-1]; if the random number is larger than the > probability value than it is dropped, otherwise it is kept. The > chances that a packet is kept is therefore 2147373991/(2**31 - 1), or > about 99.99%. This looks like optimization to reduce calculation as it is done for every packet, why not do the calculation more accurately and only for each "error", see below. > It's easy enough to fix this one, but I wasn't sure that it would be > so easy to fix the TSO/LRO issue without significantly increasing the > memory usage, so I wanted to gauge whether it was worth pursuing that > avenue or if a simpler model would be a better use of my time. The > feedback is definitely that a simpler model is *not* warranted, so > let's talk about fixing TSO/LRO. I am not even sure how to deal with TSO/LRO and BER. Your not going to discard the whole segment are you? Are you going to try and packetize it, drop the packet(s) with errors, reassmble it? My method of calculating the future error point would at least allow you to just pass a whole segment without any of that hassle and only have to do that when an error is within some segment. > > On Tue, Aug 18, 2020 at 1:47 PM Rodney W. Grimes > wrote: > > Hum, that sounds like a poor implementation indeed. It seems > > like it would be easy to convert a BER into a packet drop > > probability based on bytes that have passed through the pipe. > > I'm not quite following you; can you elaborate? Would this solution > require us to update some shared state between each packet? One > advantage of the current approach is that there is no mutable state > (except, of course, when configuration changes). You would use the bytes transfered state that is already stored, and compute a "next error" point based on BER and some randomness such that your errors are not clocked at exact BER intervals. Compare the next error point to the bytes transfered + size of this packet/segment to decide if it needs dropped, if you drop then you must calculate a new "next error" point. This should considerable reduce the overhead for error rates that effect less than 50% of packets, and would be the same overhead for BER that effect every packet. And 100x more efficient for things that effect 1% of packets. -- Rod Grimes rgri...@freebsd.org ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
On Tue, 18 Aug 2020 18:04:24 -0400 Ryan Stone wrote: > On Tue, Aug 18, 2020 at 5:43 PM Marko Zec wrote: > > Since in ng_pipe we define BER as an one error in BER bits (integer > > value), wouldn't your formula P = 1 - BER yield results less than or > > equal to zero for all non-zero values of BER? The domain of P is > > [0..1], so that won't fly. > > > > Your analysis seems to be based on an assumption that the BER > > parameter is given as a multiplier to 0.5**48, which it is not. > > > > The proper fix would be to clarify the current BER parameter > > semantics in ng_pipe(4), not to break bridges with the existing > > scripts / software which rely on ng_pipe. > > > > Cheers, > > > > Marko > > The manpage defined the BER parameter as follows: > > u_int64_t ber; /* errors per 2^48 bits */ > > If we instead want to change the documentation to match the > implementation, that's fine too. Yeah, I only saw that a few minutes ago... Fixed in r364367. Cheers Marko ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
On Tue, Aug 18, 2020 at 5:43 PM Marko Zec wrote: > Since in ng_pipe we define BER as an one error in BER bits (integer > value), wouldn't your formula P = 1 - BER yield results less than or > equal to zero for all non-zero values of BER? The domain of P is > [0..1], so that won't fly. > > Your analysis seems to be based on an assumption that the BER > parameter is given as a multiplier to 0.5**48, which it is not. > > The proper fix would be to clarify the current BER parameter semantics > in ng_pipe(4), not to break bridges with the existing scripts / software > which rely on ng_pipe. > > Cheers, > > Marko The manpage defined the BER parameter as follows: u_int64_t ber; /* errors per 2^48 bits */ If we instead want to change the documentation to match the implementation, that's fine too. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
On Tue, 18 Aug 2020 17:01:37 -0400 Ryan Stone wrote: ... > On Tue, Aug 18, 2020 at 2:56 PM Marko Zec wrote: > > The probability that a frame is completely unaffected by BER events, > > and thus shouldn't be dropped, is currently computed as > > > > Ppass(BER, plen) = Psingle_bit_unaffected(BER) ^ Nbits(plen) > > The problem is in its calculation of Psingle_bit_unaffected(BER). The > BER is the fraction of bits that are affected, therefore it is the > probability that a bit is affected. But for some reason, > Psingle_bit_unaffected(BER) is calculated as 1 - 1/BER rather than 1 - > BER. Since in ng_pipe we define BER as an one error in BER bits (integer value), wouldn't your formula P = 1 - BER yield results less than or equal to zero for all non-zero values of BER? The domain of P is [0..1], so that won't fly. Your analysis seems to be based on an assumption that the BER parameter is given as a multiplier to 0.5**48, which it is not. The proper fix would be to clarify the current BER parameter semantics in ng_pipe(4), not to break bridges with the existing scripts / software which rely on ng_pipe. Cheers, Marko > This leads to the probability table being wrong. For example, > given a BER of 2350, the probability that a 1500-byte packet is > not dropped is: > > (1 - 2350/2**48)**(1500 * 8), which is approximately 99.00%. > > However, ng_pipe calculates a fixed-point probability value of > 281460603879001. To calculate whether a frame should be dropped, > ng_pipe takes this probability value and shifts it right by 17, > yielding 2147373991. It then calls rand() to generate a random number > in the range [0,2**31-1]; if the random number is larger than the > probability value than it is dropped, otherwise it is kept. The > chances that a packet is kept is therefore 2147373991/(2**31 - 1), or > about 99.99%. > > It's easy enough to fix this one, but I wasn't sure that it would be > so easy to fix the TSO/LRO issue without significantly increasing the > memory usage, so I wanted to gauge whether it was worth pursuing that > avenue or if a simpler model would be a better use of my time. The > feedback is definitely that a simpler model is *not* warranted, so > let's talk about fixing TSO/LRO. > > On Tue, Aug 18, 2020 at 1:47 PM Rodney W. Grimes > wrote: > > Hum, that sounds like a poor implementation indeed. It seems > > like it would be easy to convert a BER into a packet drop > > probability based on bytes that have passed through the pipe. > > I'm not quite following you; can you elaborate? Would this solution > require us to update some shared state between each packet? One > advantage of the current approach is that there is no mutable state > (except, of course, when configuration changes). ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
On Tue, Aug 18, 2020 at 2:43 PM Eugene Grosbein wrote: > Sorry, missed that. But why wasn't possible? There's a daemon running on the system that handles most network configuration. It's quite inflexible and will override any manual configuration changes. It manages firewall rules but is ignorant of netgraph, so it will remove any dummynet rules but leave netgraph configuration alone. It was significantly easier to just use ng_pipe, even after having to fix or work around the bugs, than it was to fight the daemon. On Tue, Aug 18, 2020 at 2:56 PM Marko Zec wrote: > The probability that a frame is completely unaffected by BER events, > and thus shouldn't be dropped, is currently computed as > > Ppass(BER, plen) = Psingle_bit_unaffected(BER) ^ Nbits(plen) The problem is in its calculation of Psingle_bit_unaffected(BER). The BER is the fraction of bits that are affected, therefore it is the probability that a bit is affected. But for some reason, Psingle_bit_unaffected(BER) is calculated as 1 - 1/BER rather than 1 - BER. This leads to the probability table being wrong. For example, given a BER of 2350, the probability that a 1500-byte packet is not dropped is: (1 - 2350/2**48)**(1500 * 8), which is approximately 99.00%. However, ng_pipe calculates a fixed-point probability value of 281460603879001. To calculate whether a frame should be dropped, ng_pipe takes this probability value and shifts it right by 17, yielding 2147373991. It then calls rand() to generate a random number in the range [0,2**31-1]; if the random number is larger than the probability value than it is dropped, otherwise it is kept. The chances that a packet is kept is therefore 2147373991/(2**31 - 1), or about 99.99%. It's easy enough to fix this one, but I wasn't sure that it would be so easy to fix the TSO/LRO issue without significantly increasing the memory usage, so I wanted to gauge whether it was worth pursuing that avenue or if a simpler model would be a better use of my time. The feedback is definitely that a simpler model is *not* warranted, so let's talk about fixing TSO/LRO. On Tue, Aug 18, 2020 at 1:47 PM Rodney W. Grimes wrote: > Hum, that sounds like a poor implementation indeed. It seems > like it would be easy to convert a BER into a packet drop > probability based on bytes that have passed through the pipe. I'm not quite following you; can you elaborate? Would this solution require us to update some shared state between each packet? One advantage of the current approach is that there is no mutable state (except, of course, when configuration changes). ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
On Tue, 18 Aug 2020 13:17:48 -0400 Ryan Stone wrote: > I'd like to dump all of this and just implement a packet loss rate, > which would simplify all this immensely. Is anybody using ng_pipe > with a non-zero BER who would object to this? Given this litany of > issues I doubt it, but I thought that I'd be sure. Yes, the BER feature is being actively used, please don't nuke it. If you wish to supplement it with PER, which is less realistic but simpler to implement, by all means go ahead... > On Tue, Aug 18, 2020 at 1:17 PM Ryan Stone wrote: > > 4. The table calculation had two integer truncation bugs and used > > the wrong formula. I'm reasonably sure it would never calculate a > > probability other than 0 due a 64-bit constant being truncated to > > 32-bits. > > I've gone back and checked, and I was partially wrong on this point. > I had gotten the idea that integer literals would be truncated to int, > which is not true. The use of the wrong formula still means that > packets are dropped at entirely the wrong rate, though. The probability that a frame is completely unaffected by BER events, and thus shouldn't be dropped, is currently computed as Ppass(BER, plen) = Psingle_bit_unaffected(BER) ^ Nbits(plen) where Nbits(plen) = plen * 8 + user-configurable framing overhead. This is a crude model yet one which was fairly simple to implement. Could you elaborate why you consider it to be entirely wrong? Cheers, Marko ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
19.08.2020 0:17, Ryan Stone wrote: > where dummynet wasn't possible Sorry, missed that. But why wasn't possible? If you could use ng_pipe, you could probably use ng_ipfw too, or maybe create small node ng_dummynet to connect NETGRAPH network with kernel-side dummynet directly. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
19.08.2020 0:17, Ryan Stone wrote: > I'd like to dump all of this and just implement a packet loss rate, > which would simplify all this immensely. Is anybody using ng_pipe > with a non-zero BER who would object to this? Given this litany of > issues I doubt it, but I thought that I'd be sure. Take a look at dummynet(4): kldload dummynet # adds (optional) queueing delay plus 10 ms additional delay ipfw pipe 1 config bw 100Mbit/s delay 10 # add packet drop probability ipfw add 3000 prob 0.05 deny ip from any to any in # apply bandwidth limit/delay ipfw add 3010 pipe 1 ip from any to any in ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
> I recently needed to be able to simulate a lossy, high-latency network > in an environment where dummynet wasn't possible. I gave ng_pipe a > try, and hit some major issues > > 1. Instead of configuring a packet drop rate, you configure a bit > error rate, which I found significantly less intuitive >From your background being packet network centric perhaps? Those of us who have line oriented, aka telecom, centric backgrounds BER is a very meaningful and useful metric. > 2. The use of BER makes for a very inconvenient implementation, as > ng_pipe has to maintain a table of packet drop rates for every > possible packet size Hum, that sounds like a poor implementation indeed. It seems like it would be easy to convert a BER into a packet drop probability based on bytes that have passed through the pipe. It should be easy to covert a BER into a packet drop rate, but doing the converse leads to quantization errors. I would rather see us keep the BER as the metric and fix what is broken rather than convert this to a packet drop rate.. > 3. The table implementation isn't sized right for LRO or TSO, leading > to ng_pipe going out of bounds of the array and panicking the system Code predates LRO and TSO, so not unexpected. > 4. The table calculation had two integer truncation bugs and used the > wrong formula. I'm reasonably sure it would never calculate a > probability other than 0 due a 64-bit constant being truncated to > 32-bits. You retracted this. > I'd like to dump all of this and just implement a packet loss rate, > which would simplify all this immensely. Is anybody using ng_pipe > with a non-zero BER who would object to this? Given this litany of > issues I doubt it, but I thought that I'd be sure. My gut instinc is that statistically BER leads to a more realistic model. -- Rod Grimes rgri...@freebsd.org ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Is anybody using ng_pipe?
On Tue, Aug 18, 2020 at 1:17 PM Ryan Stone wrote: > 4. The table calculation had two integer truncation bugs and used the > wrong formula. I'm reasonably sure it would never calculate a > probability other than 0 due a 64-bit constant being truncated to > 32-bits. I've gone back and checked, and I was partially wrong on this point. I had gotten the idea that integer literals would be truncated to int, which is not true. The use of the wrong formula still means that packets are dropped at entirely the wrong rate, though. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Is anybody using ng_pipe?
I recently needed to be able to simulate a lossy, high-latency network in an environment where dummynet wasn't possible. I gave ng_pipe a try, and hit some major issues 1. Instead of configuring a packet drop rate, you configure a bit error rate, which I found significantly less intuitive 2. The use of BER makes for a very inconvenient implementation, as ng_pipe has to maintain a table of packet drop rates for every possible packet size 3. The table implementation isn't sized right for LRO or TSO, leading to ng_pipe going out of bounds of the array and panicking the system 4. The table calculation had two integer truncation bugs and used the wrong formula. I'm reasonably sure it would never calculate a probability other than 0 due a 64-bit constant being truncated to 32-bits. I'd like to dump all of this and just implement a packet loss rate, which would simplify all this immensely. Is anybody using ng_pipe with a non-zero BER who would object to this? Given this litany of issues I doubt it, but I thought that I'd be sure. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
