Re: [Gnoga-list] Improvements/bugfixes.
Joakim, Interesting ideas, though for this particular application pretty much unnecessary: it’s internal-use only, restricted to the internal network, and has a limited number of users who are very unlikely to be submitting at the same time. Some of these could possibly become relevant if things expanded, like the addition of instrument-scheduling / time-requests from outside sources; as of not, that’s not done (nor likely to be done). From: Joakim Dahlgren Strandberg<mailto:joaki...@kth.se> Sent: Tuesday, June 1, 2021 12:29 To: Gnoga support list<mailto:gnoga-list@lists.sourceforge.net> Subject: Re: [Gnoga-list] Improvements/bugfixes. Hi Joey, I recommend looking into the example of an https server that is part of simple components (which is included in Gnoga): components-connections_server-http_server-test_https_server.gpr Example: You could have your simple components https server at port 8080 and it has two purposes. 1. Deliver the boot.html file that delivers the empty html file but that contains the Javascript code that connects the websocket that is part of Gnoga to the port where web sockets are expected to connect. You could put this web socket at port 8081. 2. The web server can also serve as a REST API to you your web application for whenever you need REST service features like being able to make POST requests to the web server/application. I also recommend thinking about scalability and security. The Gnoga framework uses GNAT.Sockets for sockets and probably uses the select statement to know which TCP socket has received data: <https://jvns.ca/blog/2017/06/03/async-io-on-linux--select--poll--and-epoll/> Here is performance comparison (lower value is better): # operations | poll | select | epoll 10| 0.61 |0.73 | 0.41 100 | 2.9 |3.0| 0.42 1000 | 35| 35 | 0.53 1 | 990| 930 | 0.66 https://jvns.ca/blog/2017/06/03/async-io-on-linux--select--poll--and-epoll/ So if you make a web application you want to minize the number of simultaneous users of a socket. Above I wrote about having the Gnoga frameworks web-socket at port 8081. What you could have is several Gnoga web sockets, one at port 8081, another on 8082, etc. So I don't recommend just delivering the boot.html file as it is because the web socket port is hard-coded to one value there. Deliver instead a custom boot.html file where the port number has been randomized and distribute your users evenly across the available TCP web sockets at the web server. It means you achieve load balancing without having an Apache server, as I believe David Botton has done. Speaking of IT-security and Denial of Service attacks. I recommend to consider reserving a permanent web socket to each logged in user of a web site instead of having a permanent web socket to each browser that connects to the Gnoga web site. For a user that is not logged in the Gnoga web site could let the browser connect the web socket and then use it to build the HTML DOM and when finished disconnect the web socket. Unfortunately I haven't had the time to develop the ideas presented above for making a web site but I believe it is doable and I hop you enjoyed sharing my thoughts on this subject. Best regards, Joakim Från: Joey Fish Skickat: den 1 juni 2021 19:15:34 Till: Gnoga support list Ämne: Re: [Gnoga-list] Improvements/bugfixes. Blady, My use case is a online reporting-system, concerning what could be considered a multi-part/multiple-report system. Functionally, I have a collection of DIVs containing settable elements (typically FORM-elements), which can be dragged into the report’s submission area, which then reads the DIV’s elements and adds them to the FORM for submission. (This allows e.g. multiple “observation” reports to be submitted, EOD.) — The issue here comes into play when the user has a lot of notations/comments to add, as this will make the ‘message’ text-area exceed the 500 characters limitation. Much of the stuff could, eventually, be auto-generated, given an integrated redesign of the current system; this would entail much, much more and would be dealing with a very different set of operations (namely interfacing to various scientific instruments, taking the readings/data, collating & storing it; the reports in this scenario would then be essentially the metadata involved from these processes with the occasional user-notes). From: Blady via Gnoga-list<mailto:gnoga-list@lists.sourceforge.net> Sent: Saturday, May 29, 2021 10:28 To: Gnoga support list<mailto:gnoga-list@lists.sourceforge.net> Cc: Blady<mailto:blady-...@users.sf.net> Subject: Re: [Gnoga-list] Improvements/bugfixes. > Le 26 mai 2021 à 17:38, Joey Fish a écrit : > > In handling form-data it was impossible to handle, via the HTML-Form POST > method, arbitrary parameters be
Re: [Gnoga-list] Improvements/bugfixes.
Hi Joey, I recommend looking into the example of an https server that is part of simple components (which is included in Gnoga): components-connections_server-http_server-test_https_server.gpr Example: You could have your simple components https server at port 8080 and it has two purposes. 1. Deliver the boot.html file that delivers the empty html file but that contains the Javascript code that connects the websocket that is part of Gnoga to the port where web sockets are expected to connect. You could put this web socket at port 8081. 2. The web server can also serve as a REST API to you your web application for whenever you need REST service features like being able to make POST requests to the web server/application. I also recommend thinking about scalability and security. The Gnoga framework uses GNAT.Sockets for sockets and probably uses the select statement to know which TCP socket has received data: <https://jvns.ca/blog/2017/06/03/async-io-on-linux--select--poll--and-epoll/> Here is performance comparison (lower value is better): # operations | poll | select | epoll 10| 0.61 |0.73 | 0.41 100 | 2.9 |3.0| 0.42 1000 | 35| 35 | 0.53 1 | 990| 930 | 0.66 https://jvns.ca/blog/2017/06/03/async-io-on-linux--select--poll--and-epoll/ So if you make a web application you want to minize the number of simultaneous users of a socket. Above I wrote about having the Gnoga frameworks web-socket at port 8081. What you could have is several Gnoga web sockets, one at port 8081, another on 8082, etc. So I don't recommend just delivering the boot.html file as it is because the web socket port is hard-coded to one value there. Deliver instead a custom boot.html file where the port number has been randomized and distribute your users evenly across the available TCP web sockets at the web server. It means you achieve load balancing without having an Apache server, as I believe David Botton has done. Speaking of IT-security and Denial of Service attacks. I recommend to consider reserving a permanent web socket to each logged in user of a web site instead of having a permanent web socket to each browser that connects to the Gnoga web site. For a user that is not logged in the Gnoga web site could let the browser connect the web socket and then use it to build the HTML DOM and when finished disconnect the web socket. Unfortunately I haven't had the time to develop the ideas presented above for making a web site but I believe it is doable and I hop you enjoyed sharing my thoughts on this subject. Best regards, Joakim Från: Joey Fish Skickat: den 1 juni 2021 19:15:34 Till: Gnoga support list Ämne: Re: [Gnoga-list] Improvements/bugfixes. Blady, My use case is a online reporting-system, concerning what could be considered a multi-part/multiple-report system. Functionally, I have a collection of DIVs containing settable elements (typically FORM-elements), which can be dragged into the report’s submission area, which then reads the DIV’s elements and adds them to the FORM for submission. (This allows e.g. multiple “observation” reports to be submitted, EOD.) — The issue here comes into play when the user has a lot of notations/comments to add, as this will make the ‘message’ text-area exceed the 500 characters limitation. Much of the stuff could, eventually, be auto-generated, given an integrated redesign of the current system; this would entail much, much more and would be dealing with a very different set of operations (namely interfacing to various scientific instruments, taking the readings/data, collating & storing it; the reports in this scenario would then be essentially the metadata involved from these processes with the occasional user-notes). From: Blady via Gnoga-list<mailto:gnoga-list@lists.sourceforge.net> Sent: Saturday, May 29, 2021 10:28 To: Gnoga support list<mailto:gnoga-list@lists.sourceforge.net> Cc: Blady<mailto:blady-...@users.sf.net> Subject: Re: [Gnoga-list] Improvements/bugfixes. > Le 26 mai 2021 à 17:38, Joey Fish a écrit : > > In handling form-data it was impossible to handle, via the HTML-Form POST > method, arbitrary parameters because lack of a filter ensured that no > parameters were processed. Following is the fix [in source-file > gnoga-server-connection.adb] I’ve been using, though this results in some > constraints for processing which prevent multi-user simultaneous access > (because post parameters are handled individually, therefore simultaneous > submissions cannot be handled). <...> > There is another limitation, one where any particular parameter cannot exceed > 500 characters, this limit is set in the definition Gnoga_HTTP_Content, in > the same source-file. Shown below. >type Gnoga
Re: [Gnoga-list] Improvements/bugfixes.
Blady, My use case is a online reporting-system, concerning what could be considered a multi-part/multiple-report system. Functionally, I have a collection of DIVs containing settable elements (typically FORM-elements), which can be dragged into the report’s submission area, which then reads the DIV’s elements and adds them to the FORM for submission. (This allows e.g. multiple “observation” reports to be submitted, EOD.) — The issue here comes into play when the user has a lot of notations/comments to add, as this will make the ‘message’ text-area exceed the 500 characters limitation. Much of the stuff could, eventually, be auto-generated, given an integrated redesign of the current system; this would entail much, much more and would be dealing with a very different set of operations (namely interfacing to various scientific instruments, taking the readings/data, collating & storing it; the reports in this scenario would then be essentially the metadata involved from these processes with the occasional user-notes). From: Blady via Gnoga-list<mailto:gnoga-list@lists.sourceforge.net> Sent: Saturday, May 29, 2021 10:28 To: Gnoga support list<mailto:gnoga-list@lists.sourceforge.net> Cc: Blady<mailto:blady-...@users.sf.net> Subject: Re: [Gnoga-list] Improvements/bugfixes. > Le 26 mai 2021 à 17:38, Joey Fish a écrit : > > In handling form-data it was impossible to handle, via the HTML-Form POST > method, arbitrary parameters because lack of a filter ensured that no > parameters were processed. Following is the fix [in source-file > gnoga-server-connection.adb] I’ve been using, though this results in some > constraints for processing which prevent multi-user simultaneous access > (because post parameters are handled individually, therefore simultaneous > submissions cannot be handled). <...> > There is another limitation, one where any particular parameter cannot exceed > 500 characters, this limit is set in the definition Gnoga_HTTP_Content, in > the same source-file. Shown below. >type Gnoga_HTTP_Content is new Content_Source with > record > Socket : Socket_Type := null; > Connection_Type : Gnoga_Connection_Type := HTTP; > Connection_Path : Ada.Strings.Unbounded.Unbounded_String; > FS : Ada.Streams.Stream_IO.File_Type; > Input_Overflow : String_Buffer; > Buffer : String_Buffer; > Finalized : Boolean := False; > Text: aliased Strings_Edit.Streams.String_Stream (500); > end record; > > I am working on a fix for this, the simplest would be to increase the > discriminant to, say, 4096… but this merely moves the constraint to 4KB, > instead of memory arbitrary [arguments may be made for both], so I will > likely experiment on it to see if I can get a less-limited solution. Hello Joey, Well, few years ago, I came across these two issues when I had to dig in POST mechanism. I agree that this current way is not very convenient. The value 500 should be a constant in gnoga-server-connection-common.ads. Indeed, though GET and POST are supported, Gnoga is designed to play rather with events. What is your use case? Regards, Pascal. https://blady.pagesperso-orange.fr ___ Gnoga-list mailing list Gnoga-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gnoga-list ___ Gnoga-list mailing list Gnoga-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gnoga-list
Re: [Gnoga-list] Improvements/bugfixes.
> Le 26 mai 2021 à 17:38, Joey Fish a écrit : > > In handling form-data it was impossible to handle, via the HTML-Form POST > method, arbitrary parameters because lack of a filter ensured that no > parameters were processed. Following is the fix [in source-file > gnoga-server-connection.adb] I’ve been using, though this results in some > constraints for processing which prevent multi-user simultaneous access > (because post parameters are handled individually, therefore simultaneous > submissions cannot be handled). <...> > There is another limitation, one where any particular parameter cannot exceed > 500 characters, this limit is set in the definition Gnoga_HTTP_Content, in > the same source-file. Shown below. >type Gnoga_HTTP_Content is new Content_Source with > record > Socket : Socket_Type := null; > Connection_Type : Gnoga_Connection_Type := HTTP; > Connection_Path : Ada.Strings.Unbounded.Unbounded_String; > FS : Ada.Streams.Stream_IO.File_Type; > Input_Overflow : String_Buffer; > Buffer : String_Buffer; > Finalized : Boolean := False; > Text: aliased Strings_Edit.Streams.String_Stream (500); > end record; > > I am working on a fix for this, the simplest would be to increase the > discriminant to, say, 4096… but this merely moves the constraint to 4KB, > instead of memory arbitrary [arguments may be made for both], so I will > likely experiment on it to see if I can get a less-limited solution. Hello Joey, Well, few years ago, I came across these two issues when I had to dig in POST mechanism. I agree that this current way is not very convenient. The value 500 should be a constant in gnoga-server-connection-common.ads. Indeed, though GET and POST are supported, Gnoga is designed to play rather with events. What is your use case? Regards, Pascal. https://blady.pagesperso-orange.fr ___ Gnoga-list mailing list Gnoga-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gnoga-list
[Gnoga-list] Improvements/bugfixes.
In handling form-data it was impossible to handle, via the HTML-Form POST
method, arbitrary parameters because lack of a filter ensured that no
parameters were processed. Following is the fix [in source-file
gnoga-server-connection.adb] I’ve been using, though this results in some
constraints for processing which prevent multi-user simultaneous access
(because post parameters are handled individually, therefore simultaneous
submissions cannot be handled).
overriding
procedure Do_Body (Client : in out Gnoga_HTTP_Client) is
use Ada.Strings.Fixed;
use Ada.Strings.Unbounded;
use Ada.Streams.Stream_IO;
function "+"(Right : Unbounded_String) return String is
(Strings_Edit.UTF8.Handling.To_UTF8 (To_String (Right))) with Inline;
Status : Status_Line renames Get_Status_Line (Client);
Param_List : Unbounded_String;
Do_Filtering : constant Boolean := On_Post_Request_Event /= null;
Content_Type : constant String :=
Client.Get_Header (Content_Type_Header);
Disposition : constant String := Client.Get_Multipart_Header
(Content_Disposition_Header);
begin
if Do_Filtering then
On_Post_Request_Event (Status.File & Status.Query, Param_List);
end if;
if Content_Type = "application/x-www-form-urlencoded" then
if Do_Filtering then
Client.Receive_Body (+Param_List);
else
Client.Receive_Body;
end if;
end if;
if Index (Content_Type, "multipart/form-data") = Content_Type'First then
if Index (Disposition, "form-data") = Disposition'First then
declare
Field_ID : constant String := "name=""";
File_ID : constant String := "filename=""";
n : constant Natural := Index (Disposition, Field_ID);
f : constant Natural := Index (Disposition, File_ID);
begin
if n /= 0 then
declare
Eq : constant Natural := Index
(Disposition, , n + Field_ID'Length);
Field_Name : constant String := Disposition
(n + Field_ID'Length .. Eq - 1);
begin -- (not Do_Filtering) or else
if (not Do_Filtering) or else Index (+Param_List,
Field_Name) > 0 then
if f /= 0 then
declare
Eq : constant Natural := Index
(Disposition, , f + File_ID'Length);
File_Name : constant String := Disposition
(f + File_ID'Length .. Eq - 1);
begin
if On_Post_File_Event = null then
Gnoga.Log ("Attempt to upload file with out" &
" an On_Post_File_Event set");
else
if Is_Open (Client.Content.FS) then
Close (Client.Content.FS);
end if;
Create (Client.Content.FS,
Out_File,
Gnoga.Server.Upload_Directory &
File_Name & ".tmp",
"Text_Translation=No");
Receive_Body
(Client, Stream (Client.Content.FS));
On_Post_File_Event (Status.File & Status.Query,
File_Name,
File_Name & ".tmp");
end if;
end;
else
Client.Content.Text.Rewind;
Client.Receive_Body (Client.Content.Text'Access);
end if;
end if;
end;
end if;
end;
end if;
end if;
exception
when E : others =>
Log ("Do_Body Error");
Log (Ada.Exceptions.Exception_Information (E));
end Do_Body;
There is another limitation, one where any particular parameter cannot exceed
500 characters, this limit is set in the definition Gnoga_HTTP_Content, in the
same source-file. Shown below.
type Gnoga_HTTP_Content is new Content_Source with
record
Socket : Socket_Type := null;
Connection_Type : Gnoga_Connection_Type := HTTP;
Connection_Path : Ada.Strings.Unbounded.Unbounded_String;
FS : Ada.Streams.Stream_IO.File_Type;
Input_Overflow : String_Buffer;
Buffer : String_Buffer;
Finalized : Boolean :=
