Re: [U-Boot] [PATCH] TCP and wget implementation. Ptch V5 1 of 3
On 01/29/2018 06:30 PM, Duncan Hare wrote: >> The patch is trying to put everything into net.c. This is a mess and >> not where we should head to. > > Not at all. I tried that and it was correctly rejected. The TCP > functions are in tcp.c, and the wget functions in wget.c > > There is no socket. There is no socket analogue. There is no > widespread correct re-ordering of packets, because in a kernel download > the relative address of each block is derived for the tcp sequence > number, and the kernel image in memory itself is in the correct order, > as defined by tcp sequence number. You wouldn't implement a TCP protocol that cannot be reused to implement other higher protocols? TCP packets are not expected to arrive in sequence. And the length of packets may vary. You need a buffer for a few packets to put them into the right sequence again. > >> >> We should have have one driver per protocol. >> >> The IP driver should enumerate all drivers protocols like TCP and UDP >> that want to listen to it using a Linux list. This way we get rid of >> all those needless #ifdef CONFIGs. Using the list the IP driver will >> hand out packets to the respective higher protocol driver. > > The choice was to make minimal changes to the > current net.c, in consultation with other in the u-boot realm. > >> >> A separate driver shall implement the TCP protocol and provide methods >> to open a socket, to read and write to the socket and to close the >> socket. >> > "shall"? Please do not use the imperative, unless you are approaching > me with money. > >> Next we want a driver for the HTTP protocol. It should have function >> to open a connection, to send a request, to receive a response, and to >> close the connection. If this driver is requested to open a connection >> it shall call the TCP driver to open a socket. It will then receive >> all packets from the relevant IP address on the relevant port until it >> closes the socket. >> > The http protocol is very simple, and consists of a TCP connection and a > header, both for request and response. For first use implementing GET is sufficient. But the structure of the driver should allow to easily add POST and CONNECT. > >> The wget command should be in a separate file. It will call the >> appropriate functions of the HTTP driver to open a connection, post >> the request, receive the response, and finally close the connection. > > It is and it does. It also include the http headers, because of their > simplicity. > >> >> The work should start with refactoring the existing coding into >> separate drivers for the existing protocols. When that is completed >> you can start adding TCP relevant code. > > Will you pay for that work? The only thing that needs to be factored out of net.c is the IP protocol. > >> Please, do not send single patches but complete patch series. > > I do the best I can with my limited knowledge of the tools. I have sent > a series. My understand of both git and patman is limited. This is how I send a patch series: git format-patch HEAD~5 -ns --cover-letter --subject-prefix='PATCH v5' Edit the cover letter. Add in each patch after the first --- the changes relative to the prior version of the patch. Add another ---. Put only these files into a directory (let's assume it is named 'patches'). git send-email \ --from="author" \ --to="maintainer " \ --cc=u-boot@lists.denx.de \ patches\* Best regards Heinrich > >> >> Best regards >> >> Heinrich > > > > ___ > U-Boot mailing list > U-Boot@lists.denx.de > https://lists.denx.de/listinfo/u-boot > ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation. Ptch V5 1 of 3
> The patch is trying to put everything into net.c. This is a mess and > not where we should head to. Not at all. I tried that and it was correctly rejected. The TCP functions are in tcp.c, and the wget functions in wget.c There is no socket. There is no socket analogue. There is no widespread correct re-ordering of packets, because in a kernel download the relative address of each block is derived for the tcp sequence number, and the kernel image in memory itself is in the correct order, as defined by tcp sequence number. > > We should have have one driver per protocol. > > The IP driver should enumerate all drivers protocols like TCP and UDP > that want to listen to it using a Linux list. This way we get rid of > all those needless #ifdef CONFIGs. Using the list the IP driver will > hand out packets to the respective higher protocol driver. The choice was to make minimal changes to the current net.c, in consultation with other in the u-boot realm. > > A separate driver shall implement the TCP protocol and provide methods > to open a socket, to read and write to the socket and to close the > socket. > "shall"? Please do not use the imperative, unless you are approaching me with money. > Next we want a driver for the HTTP protocol. It should have function > to open a connection, to send a request, to receive a response, and to > close the connection. If this driver is requested to open a connection > it shall call the TCP driver to open a socket. It will then receive > all packets from the relevant IP address on the relevant port until it > closes the socket. > The http protocol is very simple, and consists of a TCP connection and a header, both for request and response. > The wget command should be in a separate file. It will call the > appropriate functions of the HTTP driver to open a connection, post > the request, receive the response, and finally close the connection. It is and it does. It also include the http headers, because of their simplicity. > > The work should start with refactoring the existing coding into > separate drivers for the existing protocols. When that is completed > you can start adding TCP relevant code. Will you pay for that work? > Please, do not send single patches but complete patch series. I do the best I can with my limited knowledge of the tools. I have sent a series. My understand of both git and patman is limited. > > Best regards > > Heinrich ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation. Patch V5, 3 of 3.
Hi Duncan Hare, > -Original Message- > From: U-Boot [mailto:u-boot-boun...@lists.denx.de] On Behalf Of > d...@synoia.com > Sent: Monday, January 29, 2018 9:24 AM > To: duncanch...@yahoo.com > Cc: u-boot@lists.denx.de; Joe Hershberger <joe.hershber...@ni.com>; > Maxime Ripard <maxime.rip...@free-electrons.com>; Jagan Teki > <ja...@openedev.com> > Subject: [U-Boot] [PATCH] TCP and wget implementation. Patch V5, 3 of 3. I guess you are manually adding the patch version and number. If so, I would suggest to use the following command, to allow git to prefix "PATCH v5" tag to the series and to prepare a cover letter. git format-patch --subject-prefix="PATCH v5" -3 --cover-letter While using git send-email command the following command can be used: git send-email --to=u-b...@linux.freescale.net *.patch Refer:- https://www.denx.de/wiki/U-Boot/Patches Patman is a nice tool for all this. http://git.denx.de/?p=u-boot.git;a=blob;f=tools/patman/README regards Calvin ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
[U-Boot] [PATCH] TCP and wget implementation. Patch V5, 3 of 3.
From: Duncan HareThis is the wget, tftp and Kconfig files for introducing TCP and wget into u-boot. tftp.c was enhanced to check a downloaded kernel by being able to compile it as a verification tool. The switch detween download program and tool is by c #define directive in tftp.c. Should this directive be configured by Kconfig? All the code is new, and not copied from any source. Signed-off-by: Duncan Hare --- cmd/Kconfig | 6 + cmd/net.c | 13 ++ net/tftp.c | 35 - net/wget.c | 417 net/wget.h | 22 5 files changed, 487 insertions(+), 6 deletions(-) create mode 100644 net/wget.c create mode 100644 net/wget.h diff --git a/cmd/Kconfig b/cmd/Kconfig index 5a6afab99b..46b489a966 100644 --- a/cmd/Kconfig +++ b/cmd/Kconfig @@ -1035,6 +1035,12 @@ config CMD_ETHSW operations such as enabling / disabling a port and viewing/maintaining the filtering database (FDB) +config CMD_WGET + bool "wget" + select TCP + help + Download a kernel, or other files, from a web server over TCP. + endmenu menu "Misc commands" diff --git a/cmd/net.c b/cmd/net.c index d7c776aacf..f5c2d0f8ed 100644 --- a/cmd/net.c +++ b/cmd/net.c @@ -110,6 +110,19 @@ U_BOOT_CMD( ); #endif +#if defined(CONFIG_CMD_WGET) +static int do_wget(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) +{ + return netboot_common(WGET, cmdtp, argc, argv); +} + +U_BOOT_CMD( + wget, 3, 1, do_wget, + "boot image via network using HTTP protocol", + "[loadAddress] [[hostIPaddr:]bootfilename]" +); +#endif + static void netboot_update_env(void) { char tmp[22]; diff --git a/net/tftp.c b/net/tftp.c index 6671b1f7ca..c3f8fd3053 100644 --- a/net/tftp.c +++ b/net/tftp.c @@ -6,6 +6,9 @@ *Luca Ceresoli */ +/* define if kernel load or verify kernal from a previous load */ +#define K_LOAD 1 + #include #include #include @@ -16,6 +19,9 @@ #ifdef CONFIG_SYS_DIRECT_FLASH_TFTP #include #endif +#ifndef K_LOAD +#include "tcp.h" +#endif /* Well known TFTP port # */ #define WELL_KNOWN_PORT69 @@ -29,7 +35,6 @@ #endif /* Number of "loading" hashes per line (for checking the image size) */ #define HASHES_PER_LINE65 - /* * TFTP operations. */ @@ -166,7 +171,7 @@ static void mcast_cleanup(void) static inline void store_block(int block, uchar *src, unsigned len) { - ulong offset = block * tftp_block_size + tftp_block_wrap_offset; + uint offset = block * tftp_block_size + tftp_block_wrap_offset; ulong newsize = offset + len; #ifdef CONFIG_SYS_DIRECT_FLASH_TFTP int i, rc = 0; @@ -188,14 +193,29 @@ static inline void store_block(int block, uchar *src, unsigned len) net_set_state(NETLOOP_FAIL); return; } - } else + } else { #endif /* CONFIG_SYS_DIRECT_FLASH_TFTP */ - { - void *ptr = map_sysmem(load_addr + offset, len); + void *ptr = map_sysmem(load_addr + offset, len); +#ifdef K_LOAD memcpy(ptr, src, len); +#endif +#ifndef K_LOAD + if (memcmp(ptr, src, len) != 0) { + debug_cond(1, + "File Xfer Mismatch, offset=%d, error=%d\n", + offset, memcmp(ptr, src, len)); + printf("TFTP Download Verification - Memory\n"); + tcp_print_buffer(ptr, len, len, -1, 0); + printf("TFTP Download Verification - Packet\n"); + tcp_print_buffer(src, len, len, -1, 0); + net_set_state(NETLOOP_FAIL); + } +#endif unmap_sysmem(ptr); +#ifdef CONFIG_SYS_DIRECT_FLASH_TFTP } +#endif #ifdef CONFIG_MCAST_TFTP if (tftp_mcast_active) ext2_set_bit(block, tftp_mcast_bitmap); @@ -761,6 +781,7 @@ void tftp_start(enum proto_t protocol) } printf("Using %s device\n", eth_get_name()); + printf("TFTP %s server %pI4; our IP address is %pI4", #ifdef CONFIG_CMD_TFTPPUT protocol == TFTPPUT ? "to" : "from", @@ -780,7 +801,9 @@ void tftp_start(enum proto_t protocol) printf("; sending through gateway %pI4", _gateway); } putc('\n'); - +#ifndef K_LOAD + printf("TFTP Download Verification\n"); +#endif printf("Filename '%s'.", tftp_filename); if (net_boot_file_expected_size_in_blocks) { diff --git a/net/wget.c b/net/wget.c new file mode 100644 index 00..859fd91e91 --- /dev/null +++ b/net/wget.c @@ -0,0 +1,417 @@ +/* + * FILE support driver - based on etherboot and U-BOOT's tftp.c + * + * Copyright Duncan Hare 2017 + * + * SPDX-License-Identifier:GPL-2.0+ + */
Re: [U-Boot] [PATCH] TCP and wget implementation. Ptch V5 1 of 3
On 01/29/2018 02:34 AM, Duncan Hare wrote: > Date: Sun, 28 Jan 2018 11:25:51 -0800 > > This is the interface and Kconfig files for introducing TCP and wget > into u-boot. > > Interfaces are in net.c and net.h, ping is modified to the new ip send > interface, and UDP and TCP have shim procedures call map the protocol > interface to the ip interface. > > The UDP interface is unchanged, and the existing UDP programs need no > changes. > > All the code is new, and not copied from any source. > > Makefile in the net directory is modified by hand. It appears not to be > generated by Kconfig. > > The rationale behind this change is that UDP file transfers elapsed time > is twice the sum of network latency x number of pcckets, and TCP file > transfer times are about 4x network latency plus data transit time. > > In tests this reduces kernel trnasfer time from about 15 to 20 seconds > with tftp on a raspberry pi to about 0.4 seconds with wget. > > The raspberry pi as a sink for the kernel runs at about 10 Mbits/sec. > > It is not clear if all drivers use net_rx_packets as input buffers. > Some driivers do. Some drivers also always start circular buffers use > at buffer[0]. It would be better if circlar buffer pools were used > in a truly circular manner at all times. > The patch is trying to put everything into net.c. This is a mess and not where we should head to. We should have have one driver per protocol. The IP driver should enumerate all drivers protocols like TCP and UDP that want to listen to it using a Linux list. This way we get rid of all those needless #ifdef CONFIGs. Using the list the IP driver will hand out packets to the respective higher protocol driver. A separate driver shall implement the TCP protocol and provide methods to open a socket, to read and write to the socket and to close the socket. Next we want a driver for the HTTP protocol. It should have function to open a connection, to send a request, to receive a response, and to close the connection. If this driver is requested to open a connection it shall call the TCP driver to open a socket. It will then receive all packets from the relevant IP address on the relevant port until it closes the socket. The wget command should be in a separate file. It will call the appropriate functions of the HTTP driver to open a connection, post the request, receive the response, and finally close the connection. The work should start with refactoring the existing coding into separate drivers for the existing protocols. When that is completed you can start adding TCP relevant code. Please, do not send single patches but complete patch series. Best regards Heinrich > > > Signed-off-by: Duncan Hare> --- > > include/net.h | 32 ++ > net/Kconfig | 5 +++ > net/net.c | 102 > +++--- > net/ping.c| 9 ++ > 4 files changed, 115 insertions(+), 33 deletions(-) > > diff --git a/include/net.h b/include/net.h > index 455b48f6c7..d231987e6e 100644 > --- a/include/net.h > +++ b/include/net.h > @@ -15,17 +15,26 @@ > #include > #include/* for nton* / ntoh* stuff */ > > -#define DEBUG_LL_STATE 0 /* Link local state machine changes */ > -#define DEBUG_DEV_PKT 0 /* Packets or info directed to the > device */ > -#define DEBUG_NET_PKT 0 /* Packets on info on the network at > large */ > +#define DEBUG_LL_STATE 0/* Link local state machine changes */ > +#define DEBUG_DEV_PKT 0/* Packets or info directed to the device */ > +#define DEBUG_NET_PKT 0/* Packets on info on the network at large */ > #define DEBUG_INT_STATE 0/* Internal network state changes */ > > /* > * The number of receive packet buffers, and the required packet buffer > * alignment in memory. > * > + * The nuber of buffers for TCP is used to calculate a static TCP window > + * size, becuse TCP window size is a promise to the sending TCP to be able > + * to buffer up to the window size of data. > + * When the sending TCP has a window size of outstanding unacknowledged > + * data, the sending TCP will stop sending. > */ > > +#if defined(CONFIG_TCP) > +#define CONFIG_SYS_RX_ETH_BUFFER 50 /* For TCP */ > +#endif > + > #ifdef CONFIG_SYS_RX_ETH_BUFFER > # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER > #else > @@ -354,6 +363,8 @@ struct vlan_ethernet_hdr { > > #define IPPROTO_ICMP 1 /* Internet Control Message Protocol*/ > #define IPPROTO_UDP 17 /* User Datagram Protocol */ > +#define IPPROTO_TCP 6 /* Transmission Control Protocol*/ > + > > /* > * Internet Protocol (IP) header. > @@ -538,7 +549,7 @@ extern intnet_restart_wrap; /* > Tried all network devices */ > > enum proto_t { > BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, SNTP, > - TFTPSRV, TFTPPUT, LINKLOCAL > +
[U-Boot] [PATCH] TCP and wget implementation. Ptch V5 1 of 3
Date: Sun, 28 Jan 2018 11:25:51 -0800 This is the interface and Kconfig files for introducing TCP and wget into u-boot. Interfaces are in net.c and net.h, ping is modified to the new ip send interface, and UDP and TCP have shim procedures call map the protocol interface to the ip interface. The UDP interface is unchanged, and the existing UDP programs need no changes. All the code is new, and not copied from any source. Makefile in the net directory is modified by hand. It appears not to be generated by Kconfig. The rationale behind this change is that UDP file transfers elapsed time is twice the sum of network latency x number of pcckets, and TCP file transfer times are about 4x network latency plus data transit time. In tests this reduces kernel trnasfer time from about 15 to 20 seconds with tftp on a raspberry pi to about 0.4 seconds with wget. The raspberry pi as a sink for the kernel runs at about 10 Mbits/sec. It is not clear if all drivers use net_rx_packets as input buffers. Some driivers do. Some drivers also always start circular buffers use at buffer[0]. It would be better if circlar buffer pools were used in a truly circular manner at all times. Signed-off-by: Duncan Hare--- include/net.h | 32 ++ net/Kconfig | 5 +++ net/net.c | 102 +++--- net/ping.c| 9 ++ 4 files changed, 115 insertions(+), 33 deletions(-) diff --git a/include/net.h b/include/net.h index 455b48f6c7..d231987e6e 100644 --- a/include/net.h +++ b/include/net.h @@ -15,17 +15,26 @@ #include #include /* for nton* / ntoh* stuff */ -#define DEBUG_LL_STATE 0 /* Link local state machine changes */ -#define DEBUG_DEV_PKT 0/* Packets or info directed to the device */ -#define DEBUG_NET_PKT 0/* Packets on info on the network at large */ +#define DEBUG_LL_STATE 0 /* Link local state machine changes */ +#define DEBUG_DEV_PKT 0 /* Packets or info directed to the device */ +#define DEBUG_NET_PKT 0 /* Packets on info on the network at large */ #define DEBUG_INT_STATE 0 /* Internal network state changes */ /* * The number of receive packet buffers, and the required packet buffer * alignment in memory. * + * The nuber of buffers for TCP is used to calculate a static TCP window + * size, becuse TCP window size is a promise to the sending TCP to be able + * to buffer up to the window size of data. + * When the sending TCP has a window size of outstanding unacknowledged + * data, the sending TCP will stop sending. */ +#if defined(CONFIG_TCP) +#define CONFIG_SYS_RX_ETH_BUFFER 50/* For TCP */ +#endif + #ifdef CONFIG_SYS_RX_ETH_BUFFER # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER #else @@ -354,6 +363,8 @@ struct vlan_ethernet_hdr { #define IPPROTO_ICMP1 /* Internet Control Message Protocol*/ #define IPPROTO_UDP17 /* User Datagram Protocol */ +#define IPPROTO_TCP 6 /* Transmission Control Protocol*/ + /* * Internet Protocol (IP) header. @@ -538,7 +549,7 @@ extern int net_restart_wrap; /* Tried all network devices */ enum proto_t { BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, SNTP, - TFTPSRV, TFTPPUT, LINKLOCAL + TFTPSRV, TFTPPUT, LINKLOCAL, WGET }; extern charnet_boot_file_name[1024];/* Boot File name */ @@ -596,10 +607,10 @@ int net_set_ether(uchar *xet, const uchar *dest_ethaddr, uint prot); int net_update_ether(struct ethernet_hdr *et, uchar *addr, uint prot); /* Set IP header */ -void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source); +void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source, + u16 pkt_len, u8 prot); void net_set_udp_header(uchar *pkt, struct in_addr dest, int dport, - int sport, int len); - + int sport, int len); /** * compute_ip_checksum() - Compute IP checksum * @@ -670,9 +681,16 @@ static inline void net_send_packet(uchar *pkt, int len) * @param sport Source UDP port * @param payload_len Length of data after the UDP header */ +int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport, + int payload_len, int proto, u8 action, u32 tcp_seq_num, + u32 tcp_ack_num); + int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport, int sport, int payload_len); +int net_send_tcp_packet(int payload_len, int dport, int sport, u8 action, + u32 tcp_seq_num, u32 tcp_ack_num); + /* Processes a received packet */ void net_process_received_packet(uchar *in_packet, int len); diff --git a/net/Kconfig b/net/Kconfig index 414c5497c7..625ad291bb 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -45,4 +45,9 @@
Re: [U-Boot] [PATCH] TCP and wget implementation v5.1
Hi Duncan, On Mon, Jan 15, 2018 at 11:01 AM, Duncan Harewrote: > > > > > Hi Duncan, > > On Wed, Jan 10, 2018 at 8:18 PM, Duncan Hare wrote: >> Date: Wed, 10 Jan 2018 17:54:07 -0800 >> Subject: [PATCH] git_msg_1 >> >> TCP and wget implementation. > > Sounds like this should be the title of the series cover letter. I > thought we agreed that this should be sent as a series. That means > using patman to send all 3 matches to the list as once. > > Next step is to get sendmail setup and resend as a series using patman. > >> And I do wish I could get git send-mail configureg >> with yahoo. > > https://help.yahoo.com/kb/SLN4724.html > >> >> Signed-off-by: Duncan Hare >> > > Ok, our definition of series differed. I was not thinking concurrent. I'm using common patch terminology. Please review our patch process. https://www.denx.de/wiki/U-Boot/Patches > Will send next week. > > Duncan > > ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v5.1
Hi Duncan, On Wed, Jan 10, 2018 at 8:18 PM, Duncan Harewrote: > Date: Wed, 10 Jan 2018 17:54:07 -0800 > Subject: [PATCH] git_msg_1 > > TCP and wget implementation. Sounds like this should be the title of the series cover letter. I thought we agreed that this should be sent as a series. That means using patman to send all 3 matches to the list as once. Next step is to get sendmail setup and resend as a series using patman. > And I do wish I could get git send-mail configureg > with yahoo. https://help.yahoo.com/kb/SLN4724.html > > Signed-off-by: Duncan Hare > Ok, our definition of series differed. I was not thinking concurrent. Will send next week. Duncan ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v5.1
Hi Duncan, On Wed, Jan 10, 2018 at 8:18 PM, Duncan Harewrote: > Date: Wed, 10 Jan 2018 17:54:07 -0800 > Subject: [PATCH] git_msg_1 > > TCP and wget implementation. Sounds like this should be the title of the series cover letter. I thought we agreed that this should be sent as a series. That means using patman to send all 3 matches to the list as once. > > This is the interface and Kconfig files for introducing TCP and wget > into u-boot. > > Interfaces are in net.c and net.h, ping.c is modified to the new ip send > interface, and UDP and TCP have shim procedures call map the protocol > interface to the ip interface. > > The UDP interface is unchanged, and the existing UDP programs need no > changes. > > All the code is new, and not copied from any source. > > This code should compile. > > Next step? git push? Next step is to get sendmail setup and resend as a series using patman. > And I do wish I could get git send-mail configureg > with yahoo. https://help.yahoo.com/kb/SLN4724.html > > Signed-off-by: Duncan Hare > ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
[U-Boot] [PATCH] TCP and wget implementation v5.1
Date: Wed, 10 Jan 2018 17:54:07 -0800 Subject: [PATCH] git_msg_1 TCP and wget implementation. This is the interface and Kconfig files for introducing TCP and wget into u-boot. Interfaces are in net.c and net.h, ping.c is modified to the new ip send interface, and UDP and TCP have shim procedures call map the protocol interface to the ip interface. The UDP interface is unchanged, and the existing UDP programs need no changes. All the code is new, and not copied from any source. This code should compile. Next step? git push? And I do wish I could get git send-mail configureg with yahoo. Signed-off-by: Duncan HareFrom: Duncan Hare Date: Wed, 10 Jan 2018 17:54:07 -0800 Subject: [PATCH] git_msg_1 Signed-off-by: Duncan Hare --- include/net.h | 32 ++ net/Kconfig | 5 +++ net/net.c | 102 +++--- net/ping.c| 9 ++ 4 files changed, 115 insertions(+), 33 deletions(-) diff --git a/include/net.h b/include/net.h index 455b48f6c7..d231987e6e 100644 --- a/include/net.h +++ b/include/net.h @@ -15,17 +15,26 @@ #include #include /* for nton* / ntoh* stuff */ -#define DEBUG_LL_STATE 0 /* Link local state machine changes */ -#define DEBUG_DEV_PKT 0/* Packets or info directed to the device */ -#define DEBUG_NET_PKT 0/* Packets on info on the network at large */ +#define DEBUG_LL_STATE 0 /* Link local state machine changes */ +#define DEBUG_DEV_PKT 0 /* Packets or info directed to the device */ +#define DEBUG_NET_PKT 0 /* Packets on info on the network at large */ #define DEBUG_INT_STATE 0 /* Internal network state changes */ /* * The number of receive packet buffers, and the required packet buffer * alignment in memory. * + * The nuber of buffers for TCP is used to calculate a static TCP window + * size, becuse TCP window size is a promise to the sending TCP to be able + * to buffer up to the window size of data. + * When the sending TCP has a window size of outstanding unacknowledged + * data, the sending TCP will stop sending. */ +#if defined(CONFIG_TCP) +#define CONFIG_SYS_RX_ETH_BUFFER 50/* For TCP */ +#endif + #ifdef CONFIG_SYS_RX_ETH_BUFFER # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER #else @@ -354,6 +363,8 @@ struct vlan_ethernet_hdr { #define IPPROTO_ICMP1 /* Internet Control Message Protocol*/ #define IPPROTO_UDP17 /* User Datagram Protocol */ +#define IPPROTO_TCP 6 /* Transmission Control Protocol*/ + /* * Internet Protocol (IP) header. @@ -538,7 +549,7 @@ extern int net_restart_wrap; /* Tried all network devices */ enum proto_t { BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, SNTP, - TFTPSRV, TFTPPUT, LINKLOCAL + TFTPSRV, TFTPPUT, LINKLOCAL, WGET }; extern charnet_boot_file_name[1024];/* Boot File name */ @@ -596,10 +607,10 @@ int net_set_ether(uchar *xet, const uchar *dest_ethaddr, uint prot); int net_update_ether(struct ethernet_hdr *et, uchar *addr, uint prot); /* Set IP header */ -void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source); +void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source, + u16 pkt_len, u8 prot); void net_set_udp_header(uchar *pkt, struct in_addr dest, int dport, - int sport, int len); - + int sport, int len); /** * compute_ip_checksum() - Compute IP checksum * @@ -670,9 +681,16 @@ static inline void net_send_packet(uchar *pkt, int len) * @param sport Source UDP port * @param payload_len Length of data after the UDP header */ +int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport, + int payload_len, int proto, u8 action, u32 tcp_seq_num, + u32 tcp_ack_num); + int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport, int sport, int payload_len); +int net_send_tcp_packet(int payload_len, int dport, int sport, u8 action, + u32 tcp_seq_num, u32 tcp_ack_num); + /* Processes a received packet */ void net_process_received_packet(uchar *in_packet, int len); diff --git a/net/Kconfig b/net/Kconfig index 414c5497c7..625ad291bb 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -45,4 +45,9 @@ config BOOTP_VCI_STRING default "U-Boot.arm" if ARM default "U-Boot" +config TCP + bool "Include Subset TCP stack for wget" + help + TCP protocol support for wget. + endif # if NET diff --git a/net/net.c b/net/net.c index 4259c9e321..79255c11eb 100644 --- a/net/net.c +++ b/net/net.c @@ -107,6 +107,12 @@ #if defined(CONFIG_CMD_SNTP) #include "sntp.h" #endif +#if defined(CONFIG_TCP) +#include "tcp.h" +#endif
Re: [U-Boot] [PATCH] TCP and wget implementation v4
On Fri, Jan 5, 2018 at 4:10 PM, Duncan Harewrote: >> > >> > A note on this TCP implementation. In TCP the transmitting TCP >> > guarantees delivery of a stream, and the receiving TCP guarantees >> > ordered of delivery of the stream. In this implementation The >> > kernel memory buffer and the TCP sequence number is used to order >> > the stream. for the application, and the application is the kernel >> > itself. wget is not considered the application, and does receive >> > packets "out of order." >> >> It seems like it would be possible to just store off a packet that is >> ahead of its neighbor and not call any upper handler until the needed >> packet arrives. Then all upper layers wouldn't need to know about the >> reordering. > > There is, for u-boot a large number of buffers used on RX, in order > to have a long work queue of kernel data. > > CONFIG_SYS_RX_ETH_BUFFER 50 > > The TCP transmit window is approximately 25 such packets, so overruns > are eliminated. > > There are about 3300 kernel data packets in this test kernel, so missing > a few, 3 to 5, has little impact on elapsed time, and they remain in the > input buffer pool ahead of the HTTP header, > > The only critical order for this implementation is the TCP header. > Without processing the TCP header we do not know the stream > address of the first byte of kernel data. It is easy when we know this > to map TCP stream address to kernel data offset. > >> > >> > Advice on the reset buffer approach are invited. It requires an >> > interface between the wget application to reset the buffer index. >> >> Between wget and what? The TCP implementation? It seems like something >> that should be abstracted from wget. > > wget receives packets without intermediate ordering. > Ordering data is a function of wget placing the kernel data at the > correct offset, based of TCP stream location, at the correct memory > address. > > wget is the agent which correctly orders data. There is no > socket analogue, the abstraction, and a socket's linked list buffer > reordering, which is the generally recognized reordering point for TCP > data. > > Correct ordering is a primary task of this wget implementation, > becuse the destination is a kernel image, and this choice very greatly > simplifies the TCP stack, while reducing code complexity, and limits > code size. > > Reordering would require a new piece of code similar to the fragment > assemble piece of code in net.h, and that's an amazing, but complex, > piece of code. My objective was to produce something as > simple as possible. > > The TCP stack delivers packets in the order they come off the wire. > Wget puts them in the correct order based on TCP sequence number to > build the kernel image, and the TCP sequence number/ack protocol ensures > complete delivery of a stream. OK, it sounds like you've got a solution and a preference, so if it makes more sense for this embedded implementation to maintain simplicity, then so be it. -Joe ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v4
> > > > A note on this TCP implementation. In TCP the transmitting TCP > > guarantees delivery of a stream, and the receiving TCP guarantees > > ordered of delivery of the stream. In this implementation The > > kernel memory buffer and the TCP sequence number is used to order > > the stream. for the application, and the application is the kernel > > itself. wget is not considered the application, and does receive > > packets "out of order." > > It seems like it would be possible to just store off a packet that is > ahead of its neighbor and not call any upper handler until the needed > packet arrives. Then all upper layers wouldn't need to know about the > reordering. There is, for u-boot a large number of buffers used on RX, in order to have a long work queue of kernel data. CONFIG_SYS_RX_ETH_BUFFER 50 The TCP transmit window is approximately 25 such packets, so overruns are eliminated. There are about 3300 kernel data packets in this test kernel, so missing a few, 3 to 5, has little impact on elapsed time, and they remain in the input buffer pool ahead of the HTTP header, The only critical order for this implementation is the TCP header. Without processing the TCP header we do not know the stream address of the first byte of kernel data. It is easy when we know this to map TCP stream address to kernel data offset. > > > > Advice on the reset buffer approach are invited. It requires an > > interface between the wget application to reset the buffer index. > > Between wget and what? The TCP implementation? It seems like something > that should be abstracted from wget. wget receives packets without intermediate ordering. Ordering data is a function of wget placing the kernel data at the correct offset, based of TCP stream location, at the correct memory address. wget is the agent which correctly orders data. There is no socket analogue, the abstraction, and a socket's linked list buffer reordering, which is the generally recognized reordering point for TCP data. Correct ordering is a primary task of this wget implementation, becuse the destination is a kernel image, and this choice very greatly simplifies the TCP stack, while reducing code complexity, and limits code size. Reordering would require a new piece of code similar to the fragment assemble piece of code in net.h, and that's an amazing, but complex, piece of code. My objective was to produce something as simple as possible. The TCP stack delivers packets in the order they come off the wire. Wget puts them in the correct order based on TCP sequence number to build the kernel image, and the TCP sequence number/ack protocol ensures complete delivery of a stream. ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v4
On Wed, Jan 3, 2018 at 5:01 PM, Duncan Harewrote: >> >>>selects the LIB_RAND feature since it is required. >> > >> > Thanks: will be in u-boot/cmd/Kconfig >> > >> >>Are we lookin at a series of patches, or a concurrent set? >> > >> > At this time a series of three, but I'd take advice on the preferred >> > procedure. >> >> Remember that the goal is to be atomic. >> >> You should be able to build and use U-Boot after each patch. >> >> Also, any changes to existing code that is not changing behavior but >> simply making way for new functionality should be done separately. >> Thanks >> -Joe > > A note on this TCP implementation. In TCP the transmitting TCP > guarantees delivery of a stream, and the receiving TCP guarantees > ordered of delivery of the stream. In this implementation The > kernel memory buffer and the TCP sequence number is used to order the stream. > for the application, and the application is the kernel itself. wget is > not considered the application, and does receive packets "out of order." It seems like it would be possible to just store off a packet that is ahead of its neighbor and not call any upper handler until the needed packet arrives. Then all upper layers wouldn't need to know about the reordering. > > This places a constraint on the incoming data stream. All > (disordered) packets received before the HTTP header are > ignored, which means the sending TCP will re-transmit them. This forced > re-transmission could be avoided with a change to reprocess the > incoming packet stream back to the first packet received, directly > following processing the TCP header. > > This behavior was detected and failed downloads fixed in tests > downloading Linux kernels from the cloud. > > Advice on the reset buffer approach are invited. It requires an > interface between the wget application to reset the buffer index. Between wget and what? The TCP implementation? It seems like something that should be abstracted from wget. > > Joe Thanks, Good advice, based on that the order of 3 patches is: > > (1) Prepares the interfaces, no new behavior, CONFIG-TCP is introduced. > net/Kconfig > net/net.c > net/ping.c > include/net.h > > (2) Introduces TCP > net/tcp.c > net/tcp.h > net/Makefile > > (3) Introduces wget, CONFIG-WGET introduced > cmd/Kconfig > cmd/net.c > net/wget.c > net/wget.h > > > ___ > U-Boot mailing list > U-Boot@lists.denx.de > https://lists.denx.de/listinfo/u-boot ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v4
> >>>selects the LIB_RAND feature since it is required. > > > > Thanks: will be in u-boot/cmd/Kconfig > > > >>Are we lookin at a series of patches, or a concurrent set? > > > > At this time a series of three, but I'd take advice on the preferred > > procedure. > > Remember that the goal is to be atomic. > > You should be able to build and use U-Boot after each patch. > > Also, any changes to existing code that is not changing behavior but > simply making way for new functionality should be done separately. > Thanks > -Joe A note on this TCP implementation. In TCP the transmitting TCP guarantees delivery of a stream, and the receiving TCP guarantees ordered of delivery of the stream. In this implementation The kernel memory buffer and the TCP sequence number is used to order the stream. for the application, and the application is the kernel itself. wget is not considered the application, and does receive packets "out of order." This places a constraint on the incoming data stream. All (disordered) packets received before the HTTP header are ignored, which means the sending TCP will re-transmit them. This forced re-transmission could be avoided with a change to reprocess the incoming packet stream back to the first packet received, directly following processing the TCP header. This behavior was detected and failed downloads fixed in tests downloading Linux kernels from the cloud. Advice on the reset buffer approach are invited. It requires an interface between the wget application to reset the buffer index. Joe Thanks, Good advice, based on that the order of 3 patches is: (1) Prepares the interfaces, no new behavior, CONFIG-TCP is introduced. net/Kconfig net/net.c net/ping.c include/net.h (2) Introduces TCP net/tcp.c net/tcp.h net/Makefile (3) Introduces wget, CONFIG-WGET introduced cmd/Kconfig cmd/net.c net/wget.c net/wget.h ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v3
On Wed, Jan 3, 2018 at 3:07 PM, Duncan Harewrote: > >>>selects the LIB_RAND feature since it is required. > > Thanks: will be in u-boot/cmd/Kconfig > >>Are we lookin at a series of patches, or a concurrent set? > > At this time a series of three, but I'd take advice on the preferred > procedure. Remember that the goal is to be atomic. You should be able to build and use U-Boot after each patch. For instance, if you are adding a new command, the source for that command, the Kconfig option, the Makefile change to build it, etc should be in the same commit. Similarly, each commit should do as little as is reasonably a distinct change. Also, any changes to existing code that is not changing behavior but simply making way for new functionality should be done separately. This way it's easier to review in the mindset of "this should be functionally equivalent", making it easier to spot issues without getting lost in a ton of code. Thanks, -Joe > (1) > cmd/Kconfig > cmd/net.c > > net/Kconfig > net/net.c > net/ping.c > > include/net.h > > (2) > net/tcp.c > net/tcp.h > net/Makefile > > (3) > net/wget.c > net/wget.h > > ___ > U-Boot mailing list > U-Boot@lists.denx.de > https://lists.denx.de/listinfo/u-boot ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v3
>>selects the LIB_RAND feature since it is required. Thanks: will be in u-boot/cmd/Kconfig >Are we lookin at a series of patches, or a concurrent set? At this time a series of three, but I'd take advice on the preferred procedure. (1) cmd/Kconfig cmd/net.c net/Kconfig net/net.c net/ping.c include/net.h (2) net/tcp.c net/tcp.h net/Makefile (3) net/wget.c net/wget.h ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Re: [U-Boot] [PATCH] TCP and wget implementation v3
On Tue, Dec 19, 2017 at 11:02 AM, Duncan Harewrote: >>I'd like to see the shim and all changes to existing stack as a >>separate patch to make review simpler. > > In this patch. > >>Also please make TCP and wget into 2 other separate patches. > > Yes, following. > >> Yes it should... you could have wget "selects" tcp. > Please send a "how to" example. Line 12 in net/Kconfig is an example. The NET_RANDOM_ETHADDR feature selects the LIB_RAND feature since it is required. > >>> The rationale behind this change is that UDP file transfers elapsed >>> time is twice the sum of network latency x number of pcckets, and >>> TCP file transfer times are about 4x network latency plus data >>> transit time. > >>What settings are used in this tftp benchmark for block size? It makes >>a huge difference to the performance. > > We use the default blocksize. > > We decided on TCP becuse it is probably the best high performance, > reliable protocol available, and the simplicity of the http protocol. > > UDP has its place. Relaible fast file transfer is a poor fit for UDP. > > We we lookin at a series of patches, or a concurrent set? The > previous patch include all the pvarius change for setting up > integration into u-boot. This patch standing by itself cannot > be integrated. At a minimun it needs the slightly changed > version of ping. > > Signed-off-by: Duncan Hare > --- > > include/net.h | 32 ++ > net/net.c | 102 > +++--- 2 files > changed, 108 insertions(+), 26 deletions(-) > > diff --git a/include/net.h b/include/net.h > index 455b48f6c7..d231987e6e 100644 > --- a/include/net.h > +++ b/include/net.h > @@ -15,17 +15,26 @@ > #include > #include /* for nton* / ntoh* stuff */ > > -#define DEBUG_LL_STATE 0 /* Link local state machine changes */ > -#define DEBUG_DEV_PKT 0/* Packets or info directed to > the device */ -#define DEBUG_NET_PKT 0 /* Packets on > info on the network at large */ +#define DEBUG_LL_STATE 0 /* > Link local state machine changes */ +#define DEBUG_DEV_PKT > 0 /* Packets or info directed to the device */ +#define > DEBUG_NET_PKT 0 /* Packets on info on the network at large */ > #define DEBUG_INT_STATE 0 /* Internal network state changes */ > /* > * The number of receive packet buffers, and the required > packet buffer > * alignment in memory. > * > + * The nuber of buffers for TCP is used to calculate a static > TCP window > + * size, becuse TCP window size is a promise to the sending TCP > to be able > + * to buffer up to the window size of data. > + * When the sending TCP has a window size of outstanding > unacknowledged > + * data, the sending TCP will stop sending. > */ > > +#if defined(CONFIG_TCP) > +#define CONFIG_SYS_RX_ETH_BUFFER 50/* For TCP */ > +#endif > + > #ifdef CONFIG_SYS_RX_ETH_BUFFER > # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER > #else > @@ -354,6 +363,8 @@ struct vlan_ethernet_hdr { > > #define IPPROTO_ICMP1 /* Internet Control Message > Protocol*/ #define IPPROTO_UDP 17 /* User > Datagram Protocol */ +#define IPPROTO_TCP > 6 /* Transmission Control Protocol*/ + > > /* > * Internet Protocol (IP) header. > @@ -538,7 +549,7 @@ extern int > net_restart_wrap; /* Tried all network devices */ > enum proto_t { > BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, > SNTP, > - TFTPSRV, TFTPPUT, LINKLOCAL > + TFTPSRV, TFTPPUT, LINKLOCAL, WGET > }; > > extern charnet_boot_file_name[1024];/* Boot File name */ > @@ -596,10 +607,10 @@ int net_set_ether(uchar *xet, const uchar > *dest_ethaddr, uint prot); int net_update_ether(struct ethernet_hdr > *et, uchar *addr, uint prot); > /* Set IP header */ > -void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr > source); +void net_set_ip_header(uchar *pkt, struct in_addr dest, > struct in_addr source, > + u16 pkt_len, u8 prot); > void net_set_udp_header(uchar *pkt, struct in_addr dest, int dport, > - int sport, int len); > - > + int sport, int len); > /** > * compute_ip_checksum() - Compute IP checksum > * > @@ -670,9 +681,16 @@ static inline void net_send_packet(uchar *pkt, int > len) > * @param sport Source UDP port > * @param payload_len Length of data after the UDP header > */ > +int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, > int sport, > + int payload_len, int proto, u8 action, u32 > tcp_seq_num, > + u32 tcp_ack_num); > + > int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport, > int sport, int payload_len); > > +int net_send_tcp_packet(int payload_len, int dport, int sport, u8 > action, > + u32
[U-Boot] [PATCH] TCP and wget implementation v3
>I'd like to see the shim and all changes to existing stack as a >separate patch to make review simpler. In this patch. >Also please make TCP and wget into 2 other separate patches. Yes, following. > Yes it should... you could have wget "selects" tcp. Please send a "how to" example. >> The rationale behind this change is that UDP file transfers elapsed >> time is twice the sum of network latency x number of pcckets, and >> TCP file transfer times are about 4x network latency plus data >> transit time. >What settings are used in this tftp benchmark for block size? It makes >a huge difference to the performance. We use the default blocksize. We decided on TCP becuse it is probably the best high performance, reliable protocol available, and the simplicity of the http protocol. UDP has its place. Relaible fast file transfer is a poor fit for UDP. We we lookin at a series of patches, or a concurrent set? The previous patch include all the pvarius change for setting up integration into u-boot. This patch standing by itself cannot be integrated. At a minimun it needs the slightly changed version of ping. Signed-off-by: Duncan Hare--- include/net.h | 32 ++ net/net.c | 102 +++--- 2 files changed, 108 insertions(+), 26 deletions(-) diff --git a/include/net.h b/include/net.h index 455b48f6c7..d231987e6e 100644 --- a/include/net.h +++ b/include/net.h @@ -15,17 +15,26 @@ #include #include /* for nton* / ntoh* stuff */ -#define DEBUG_LL_STATE 0 /* Link local state machine changes */ -#define DEBUG_DEV_PKT 0/* Packets or info directed to the device */ -#define DEBUG_NET_PKT 0 /* Packets on info on the network at large */ +#define DEBUG_LL_STATE 0 /* Link local state machine changes */ +#define DEBUG_DEV_PKT 0 /* Packets or info directed to the device */ +#define DEBUG_NET_PKT 0 /* Packets on info on the network at large */ #define DEBUG_INT_STATE 0 /* Internal network state changes */ /* * The number of receive packet buffers, and the required packet buffer * alignment in memory. * + * The nuber of buffers for TCP is used to calculate a static TCP window + * size, becuse TCP window size is a promise to the sending TCP to be able + * to buffer up to the window size of data. + * When the sending TCP has a window size of outstanding unacknowledged + * data, the sending TCP will stop sending. */ +#if defined(CONFIG_TCP) +#define CONFIG_SYS_RX_ETH_BUFFER 50/* For TCP */ +#endif + #ifdef CONFIG_SYS_RX_ETH_BUFFER # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER #else @@ -354,6 +363,8 @@ struct vlan_ethernet_hdr { #define IPPROTO_ICMP1 /* Internet Control Message Protocol*/ #define IPPROTO_UDP 17 /* User Datagram Protocol */ +#define IPPROTO_TCP 6 /* Transmission Control Protocol*/ + /* * Internet Protocol (IP) header. @@ -538,7 +549,7 @@ extern int net_restart_wrap; /* Tried all network devices */ enum proto_t { BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, SNTP, - TFTPSRV, TFTPPUT, LINKLOCAL + TFTPSRV, TFTPPUT, LINKLOCAL, WGET }; extern charnet_boot_file_name[1024];/* Boot File name */ @@ -596,10 +607,10 @@ int net_set_ether(uchar *xet, const uchar *dest_ethaddr, uint prot); int net_update_ether(struct ethernet_hdr *et, uchar *addr, uint prot); /* Set IP header */ -void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source); +void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source, + u16 pkt_len, u8 prot); void net_set_udp_header(uchar *pkt, struct in_addr dest, int dport, - int sport, int len); - + int sport, int len); /** * compute_ip_checksum() - Compute IP checksum * @@ -670,9 +681,16 @@ static inline void net_send_packet(uchar *pkt, int len) * @param sport Source UDP port * @param payload_len Length of data after the UDP header */ +int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport, + int payload_len, int proto, u8 action, u32 tcp_seq_num, + u32 tcp_ack_num); + int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport, int sport, int payload_len); +int net_send_tcp_packet(int payload_len, int dport, int sport, u8 action, + u32 tcp_seq_num, u32 tcp_ack_num); + /* Processes a received packet */ void net_process_received_packet(uchar *in_packet, int len); diff --git a/net/net.c b/net/net.c index 4259c9e321..79255c11eb 100644 --- a/net/net.c +++ b/net/net.c @@ -107,6 +107,12 @@ #if defined(CONFIG_CMD_SNTP) #include "sntp.h" #endif +#if defined(CONFIG_TCP) +#include "tcp.h" +#endif +#if defined(CONFIG_CMD_WGET) +#include "wget.h"
Re: [U-Boot] [PATCH] TCP and wget implementation.
Hi Duncan, The subject should not end with a '.' character. Also, this should have been marked as v2. Please mark the next as v3. On Wed, Nov 8, 2017 at 6:34 PM, Duncan Harewrote: > his is the interface and Kconfig files for introducing TCP and wget > into u-boot. Missing 'T' at the beginning? > Interfaces are in net.c and net.h, ping is modified to the new ip send > interface, and UDP and TCP have shim procedures call map the protocol > interface to the ip interface. I'd like to see the shim and all changes to existing stack as a separate patch to make review simpler. Also please make TCP and wget into 2 other separate patches. > The UDP interface is unchanged, and the existing UDP programs need no > changes. > > All the code is new, and not copied from any source. > > The wget command and TCP stack are separately configured in Kconfig, > and provisioning wget without TCP will result in a number of error > messages. It might be possible to have Kconfig handle dependencies, > if so advice is welcome. Yes it should... you could have wget "selects" tcp. > Makefile in the net directory is modified by hand. It appears not to be > generated by Kconfig. Again advice is welcome. That's fine. > The rationale behind this change is that UDP file transfers elapsed time > is twice the sum of network latency x number of pcckets, and TCP file > transfer times are about 4x network latency plus data transit time. > > In tests this reduces kernel trnasfer time from about 15 to 20 seconds > with tftp on a raspberry pi to about 0.4 seconds with wget. What settings are used in this tftp benchmark for block size? It makes a huge difference to the performance. > The raspberry pi as a sink for the kernel runs at about 10 Mbits/sec. > > Signed-off-by: Duncan Hare > --- > > cmd/Kconfig | 5 +++ > cmd/net.c | 13 > include/net.h | 26 --- > net/Kconfig | 5 +++ > net/Makefile | 3 +- > net/net.c | 100 > -- > net/ping.c| 9 ++ > 7 files changed, 132 insertions(+), 29 deletions(-) > > diff --git a/cmd/Kconfig b/cmd/Kconfig > index 5a6afab99b..4e5bac685e 100644 > --- a/cmd/Kconfig > +++ b/cmd/Kconfig > @@ -1035,6 +1035,11 @@ config CMD_ETHSW > operations such as enabling / disabling a port and > viewing/maintaining the filtering database (FDB) > > +config CMD_WGET > + bool "wget" > + help > + Download a kernel, or other files, from a web server over TCP. > + > endmenu > > menu "Misc commands" > diff --git a/cmd/net.c b/cmd/net.c > index d7c776aacf..f5c2d0f8ed 100644 > --- a/cmd/net.c > +++ b/cmd/net.c > @@ -110,6 +110,19 @@ U_BOOT_CMD( > ); > #endif > > +#if defined(CONFIG_CMD_WGET) > +static int do_wget(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) > +{ > + return netboot_common(WGET, cmdtp, argc, argv); > +} > + > +U_BOOT_CMD( > + wget, 3, 1, do_wget, > + "boot image via network using HTTP protocol", > + "[loadAddress] [[hostIPaddr:]bootfilename]" > +); > +#endif > + > static void netboot_update_env(void) > { > char tmp[22]; > diff --git a/include/net.h b/include/net.h > index 455b48f6c7..7787413816 100644 > --- a/include/net.h > +++ b/include/net.h > @@ -24,8 +24,17 @@ > * The number of receive packet buffers, and the required packet buffer > * alignment in memory. > * > + * The nuber of buffers for TCP is used to calculate a static TCP window > + * size, becuse TCP window size is a promise to the sending TCP to be > able > + * to buffer up to the window size of data. > + * When the sending TCP has a window size of outstanding unacknowledged > + * data, the sending TCP will stop sending. > */ > > +#if defined(CONFIG_TCP) > +#define CONFIG_SYS_RX_ETH_BUFFER 50/* For TCP */ > +#endif > + > #ifdef CONFIG_SYS_RX_ETH_BUFFER > # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER > #else > @@ -354,6 +363,8 @@ struct vlan_ethernet_hdr { > > #define IPPROTO_ICMP1 /* Internet Control Message Protocol*/ > #define IPPROTO_UDP17 /* User Datagram Protocol */ > +#define IPPROTO_TCP 6 /* Transmission Control Protocol*/ > + > > /* > * Internet Protocol (IP) header. > @@ -538,7 +549,7 @@ extern int net_restart_wrap; /* Tried all > network devices */ > > enum proto_t { > BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, SNTP, > - TFTPSRV, TFTPPUT, LINKLOCAL > + TFTPSRV, TFTPPUT, LINKLOCAL, WGET > }; > > extern charnet_boot_file_name[1024];/* Boot File name */ > @@ -596,10 +607,10 @@ int net_set_ether(uchar *xet, const uchar > *dest_ethaddr, uint prot); > int net_update_ether(struct ethernet_hdr *et, uchar *addr, uint prot); > > /* Set IP header */ > -void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr > source);
[U-Boot] [PATCH] TCP and wget implementation.
The changes in u-boot to implement this were sent Nov 8, and I've see no mention of it, in the mailing list. What's the next step? Thanks Duncan Hare d...@synoia.com ___ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
[U-Boot] [PATCH] TCP and wget implementation.
his is the interface and Kconfig files for introducing TCP and wget into u-boot. Interfaces are in net.c and net.h, ping is modified to the new ip send interface, and UDP and TCP have shim procedures call map the protocol interface to the ip interface. The UDP interface is unchanged, and the existing UDP programs need no changes. All the code is new, and not copied from any source. The wget command and TCP stack are separately configured in Kconfig, and provisioning wget without TCP will result in a number of error messages. It might be possible to have Kconfig handle dependencies, if so advice is welcome. Makefile in the net directory is modified by hand. It appears not to be generated by Kconfig. Again advice is welcome. The rationale behind this change is that UDP file transfers elapsed time is twice the sum of network latency x number of pcckets, and TCP file transfer times are about 4x network latency plus data transit time. In tests this reduces kernel trnasfer time from about 15 to 20 seconds with tftp on a raspberry pi to about 0.4 seconds with wget. The raspberry pi as a sink for the kernel runs at about 10 Mbits/sec. Signed-off-by: Duncan Hare--- cmd/Kconfig | 5 +++ cmd/net.c | 13 include/net.h | 26 --- net/Kconfig | 5 +++ net/Makefile | 3 +- net/net.c | 100 -- net/ping.c| 9 ++ 7 files changed, 132 insertions(+), 29 deletions(-) diff --git a/cmd/Kconfig b/cmd/Kconfig index 5a6afab99b..4e5bac685e 100644 --- a/cmd/Kconfig +++ b/cmd/Kconfig @@ -1035,6 +1035,11 @@ config CMD_ETHSW operations such as enabling / disabling a port and viewing/maintaining the filtering database (FDB) +config CMD_WGET + bool "wget" + help + Download a kernel, or other files, from a web server over TCP. + endmenu menu "Misc commands" diff --git a/cmd/net.c b/cmd/net.c index d7c776aacf..f5c2d0f8ed 100644 --- a/cmd/net.c +++ b/cmd/net.c @@ -110,6 +110,19 @@ U_BOOT_CMD( ); #endif +#if defined(CONFIG_CMD_WGET) +static int do_wget(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) +{ + return netboot_common(WGET, cmdtp, argc, argv); +} + +U_BOOT_CMD( + wget, 3, 1, do_wget, + "boot image via network using HTTP protocol", + "[loadAddress] [[hostIPaddr:]bootfilename]" +); +#endif + static void netboot_update_env(void) { char tmp[22]; diff --git a/include/net.h b/include/net.h index 455b48f6c7..7787413816 100644 --- a/include/net.h +++ b/include/net.h @@ -24,8 +24,17 @@ * The number of receive packet buffers, and the required packet buffer * alignment in memory. * + * The nuber of buffers for TCP is used to calculate a static TCP window + * size, becuse TCP window size is a promise to the sending TCP to be able + * to buffer up to the window size of data. + * When the sending TCP has a window size of outstanding unacknowledged + * data, the sending TCP will stop sending. */ +#if defined(CONFIG_TCP) +#define CONFIG_SYS_RX_ETH_BUFFER 50/* For TCP */ +#endif + #ifdef CONFIG_SYS_RX_ETH_BUFFER # define PKTBUFSRX CONFIG_SYS_RX_ETH_BUFFER #else @@ -354,6 +363,8 @@ struct vlan_ethernet_hdr { #define IPPROTO_ICMP1 /* Internet Control Message Protocol*/ #define IPPROTO_UDP17 /* User Datagram Protocol */ +#define IPPROTO_TCP 6 /* Transmission Control Protocol*/ + /* * Internet Protocol (IP) header. @@ -538,7 +549,7 @@ extern int net_restart_wrap; /* Tried all network devices */ enum proto_t { BOOTP, RARP, ARP, TFTPGET, DHCP, PING, DNS, NFS, CDP, NETCONS, SNTP, - TFTPSRV, TFTPPUT, LINKLOCAL + TFTPSRV, TFTPPUT, LINKLOCAL, WGET }; extern charnet_boot_file_name[1024];/* Boot File name */ @@ -596,10 +607,10 @@ int net_set_ether(uchar *xet, const uchar *dest_ethaddr, uint prot); int net_update_ether(struct ethernet_hdr *et, uchar *addr, uint prot); /* Set IP header */ -void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source); +void net_set_ip_header(uchar *pkt, struct in_addr dest, struct in_addr source, + u16 pkt_len, u8 prot); void net_set_udp_header(uchar *pkt, struct in_addr dest, int dport, - int sport, int len); - + int sport, int len); /** * compute_ip_checksum() - Compute IP checksum * @@ -670,9 +681,16 @@ static inline void net_send_packet(uchar *pkt, int len) * @param sport Source UDP port * @param payload_len Length of data after the UDP header */ +int net_send_ip_packet(uchar *ether, struct in_addr dest, int dport, int sport, + int payload_len, int proto, u8 action, u32 tcp_seq_num, + u32 tcp_ack_num); + int net_send_udp_packet(uchar *ether, struct in_addr dest, int dport,