Re: [racket-users] Escaping strings for the shell
Well, that sounds pretty fantastic. Looks like it’s time to take another look at Rash! John > On Jan 15, 2019, at 1:40 PM, William G Hatch wrote: > > On Tue, Jan 15, 2019 at 04:11:39PM -0500, 'John Clements' via Racket Users > wrote: >> Does rash have autocompletion of paths, yet? That’s my one super-super >> wishlist item. > > It has some basic path autocompletion -- if you type `ls /e` and hit > tab it will complete to `ls /etc/`, and you can hit tab again for a > list of files/directories in /etc. But it doesn't see through > variables or anything that needs to be evaluated, and completion is > not really very programmable at all. > >>> On Jan 15, 2019, at 6:17 AM, 'Paulo Matos' via Racket Users >>> wrote: >>> >>> I am surprised nobody mentioned Rash. I have been using it for all my >>> shell scripting needs and it's awesome. >>> >>> https://pkgs.racket-lang.org/package/rash >>> >>> On 29/12/2018 05:09, David Storrs wrote: I am using 'system' to offload some work onto wget and other applications in a few one-off scripts. Is there an easy way to escape a string so it's suitable for usage in the shell? Things like backwhacking all the quotes and relevant spaces and such. >>> >>> -- >>> Paulo Matos >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Racket Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to racket-users+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Racket Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to racket-users+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
On Tue, Jan 15, 2019 at 04:11:39PM -0500, 'John Clements' via Racket Users wrote: Does rash have autocompletion of paths, yet? That’s my one super-super wishlist item. It has some basic path autocompletion -- if you type `ls /e` and hit tab it will complete to `ls /etc/`, and you can hit tab again for a list of files/directories in /etc. But it doesn't see through variables or anything that needs to be evaluated, and completion is not really very programmable at all. On Jan 15, 2019, at 6:17 AM, 'Paulo Matos' via Racket Users wrote: I am surprised nobody mentioned Rash. I have been using it for all my shell scripting needs and it's awesome. https://pkgs.racket-lang.org/package/rash On 29/12/2018 05:09, David Storrs wrote: I am using 'system' to offload some work onto wget and other applications in a few one-off scripts. Is there an easy way to escape a string so it's suitable for usage in the shell? Things like backwhacking all the quotes and relevant spaces and such. -- Paulo Matos -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
Does rash have autocompletion of paths, yet? That’s my one super-super wishlist item. > On Jan 15, 2019, at 6:17 AM, 'Paulo Matos' via Racket Users > wrote: > > I am surprised nobody mentioned Rash. I have been using it for all my > shell scripting needs and it's awesome. > > https://pkgs.racket-lang.org/package/rash > > On 29/12/2018 05:09, David Storrs wrote: >> I am using 'system' to offload some work onto wget and other >> applications in a few one-off scripts. Is there an easy way to escape >> a string so it's suitable for usage in the shell? Things like >> backwhacking all the quotes and relevant spaces and such. >> > > -- > Paulo Matos > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
Oh, neat. Thank you. On Tue, Jan 15, 2019 at 9:18 AM 'Paulo Matos' via Racket Users wrote: > > I am surprised nobody mentioned Rash. I have been using it for all my > shell scripting needs and it's awesome. > > https://pkgs.racket-lang.org/package/rash > > On 29/12/2018 05:09, David Storrs wrote: > > I am using 'system' to offload some work onto wget and other > > applications in a few one-off scripts. Is there an easy way to escape > > a string so it's suitable for usage in the shell? Things like > > backwhacking all the quotes and relevant spaces and such. > > > > -- > Paulo Matos > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
I am surprised nobody mentioned Rash. I have been using it for all my shell scripting needs and it's awesome. https://pkgs.racket-lang.org/package/rash On 29/12/2018 05:09, David Storrs wrote: > I am using 'system' to offload some work onto wget and other > applications in a few one-off scripts. Is there an easy way to escape > a string so it's suitable for usage in the shell? Things like > backwhacking all the quotes and relevant spaces and such. > -- Paulo Matos -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
I agree with all of that -- it's definitely not a desirable thing on a general basis. For this single use, with a well-defined use case and known parameters, it would have been useful, but it's not critical. Thanks for clarifying about the system* thing with putting -O in a separate argument. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
In a normal command line I would put single quotes around that and it would be fine, or backwhack the /. Unfortunately, Racket disliked both of those solutions, hence the problem. That seems strange, if you're on a Unix like GNU/Linux or a BSD. I might be misunderstanding, or maybe Racket is doing something that I'd find surprising (and is perhaps a bug), or maybe we should clarify for the list... When you got a forward-slash in a filename (not a pathname), was it on MS Windows or in a particular Mac OS X interface context? Or maybe it wasn't a ASCII character 47 (decimal), but a Unicode lookalike glyph? Or maybe some other problem was confusing things? On a Unix, I assume that Racket `process*`-like procedures will pass through the argument strings to the OS system call API, with at most character encoding translation, but not interpret escaping/quoting. On a Unix, you really-really shouldn't be able to put a forward-slash (ASCII 47) in a filename. Even if the kernel/filesystem would permit it, that would break other things throughout the system. That might even create a new class of security exploits (e.g., create a filename that uses "/", "..", and/or ".", which a later privileged process interprets differently, to mutate/delete another file, under your control). (And, if it's different forward-slash Unicode character, it's asking to confuse a human in the loop, like a lookalike-glyph impostor banking Web site domain name.) Even if you got one filesystem to support it on a particular platform, on most other filesystems/protocols that OS might also end up using (e.g., mounted removable USB drives for sneakernet with other systems, some network filesystems, simple cloud storage mapping like in AWS S3) something would likely to break. I just read that at least one Mac OS X filesystem, at a low level, permits arbitrary characters in a filename, but that they prevent some problematic characters (including forward-slash) through most interfaces. This sounds like a good call, to me. Given the frequent desire to leverage the huge wealth of existing Posix-assuming stuff and user understanding, and to interoperate and possibly be portable... Even when we can put ASCII 47 forward-slash (and other especially problematic characters) in a filename on a particular platform, through a particular interface... usually we want to avoid that, and design our systems from the start not to do that. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
On Sat, Dec 29, 2018 at 2:25 PM Neil Van Dyke wrote: > > > > /tmp/Sara's birthday, 9/12/01-18/181: No such file or directory > > The "/" in the filename is a separate problem, which actually has > nothing to do with shell special character command line > escaping/quoting. If you're on a Unix derivative OS, it normally > expects "/" to be a directory separator in a file pathname Yes. That's why I expected it to fail. :> In a normal command line I would put single quotes around that and it would be fine, or backwhack the /. Unfortunately, Racket disliked both of those solutions, hence the problem. I'm trying to download a friend's files and was trying to use the filenames she gave them on Flickr. I already had a working version that transformed the filenames to remove difficult characters, but I was hoping there was a cleaner way. Thank you for the help -- I hadn't realized the bit about needing -O to be separate. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
On 2018-12-29 19:08, David Storrs wrote: On Fri, Dec 28, 2018 at 11:34 PM Jack Rosenthal wrote: On Fri, 28 Dec 2018 at 23:09 -0500, David Storrs wrote: > I am using 'system' to offload some work onto wget and other > applications in a few one-off scripts. Is there an easy way to escape > a string so it's suitable for usage in the shell? Things like > backwhacking all the quotes and relevant spaces and such. Is avoiding the shell altogether an option? You could use "system*" Not really: Welcome to Racket v6.11. ; two argument strings (system* "/usr/local/bin/wget" "http://google.com; "-O /tmp/fooflaksdjdghk") /tmp/fooflaksdjdghk: No such file or directory #f ; one argument string (system* "/usr/local/bin/wget" "http://google.com -O /tmp/fooflaksdjdghk") --13:47:10-- http://google.com%20-o%20/tmp/fooflaksdjdghk => `fooflaksdjdghk' Resolving google.com -o ... failed: nodename nor servname provided, or not known. #f Okay, it's transforming -O into -o. Let's try the long form: (system* "/usr/local/bin/wget" "http://google.com; "--output-document=/tmp/zot") ...lots of spew... #t Yep, that works. Okay, now let's try it with an actual filename, unescaped to start. I expect this to fail: (system* "/usr/local/bin/wget" "http://google.com; "--output-document=/tmp/Sara's birthday, 9/12/01-18/181") /tmp/Sara's birthday, 9/12/01-18/181: No such file or directory #f Sure enough. Oh good, we're back to shell-style escapes where all this started. You just did it wrong. SYSTEM* is actually useful in this situation. (system* "/usr/local/bin/wget" "http://google.com; "-O" "/tmp/fooflaksdjdghk") The -o needed to be a separate argv parameter. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
(system* "/usr/local/bin/wget" "http://google.com; "-O /tmp/fooflaksdjdghk") The process wants "-O" here to be an argument by itself. Okay, it's transforming -O into -o. Let's try the long form: That happened because `wget` thought that you were still in the server authority host part of the URL, which in alphabetic case-insensitive, and it's normalizing the case. :) Yep, that works. Okay, now let's try it with an actual filename, Yes, that's a good way to do it. Or one argument for "-O" and another for the filename. /tmp/Sara's birthday, 9/12/01-18/181: No such file or directory The "/" in the filename is a separate problem, which actually has nothing to do with shell special character command line escaping/quoting. If you're on a Unix derivative OS, it normally expects "/" to be a directory separator in a file pathname (although maybe your OS does something different), so it thinks you're talking about a file under the "181" directory, which doesn't exist. So this error is due to something that is happening within or under the `wget` process (which is very much a Unix tool), either in its own code, in filesystem routines it uses, or in the OS services it's calling. Probably you don't want to put "/" in filenames, because, even if your current OS supports it, most filesystems and cloud storage interfaces will not. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
On Fri, Dec 28, 2018 at 11:34 PM Jack Rosenthal wrote: > > On Fri, 28 Dec 2018 at 23:09 -0500, David Storrs wrote: > > I am using 'system' to offload some work onto wget and other > > applications in a few one-off scripts. Is there an easy way to escape > > a string so it's suitable for usage in the shell? Things like > > backwhacking all the quotes and relevant spaces and such. > > Is avoiding the shell altogether an option? You could use "system*" Not really: Welcome to Racket v6.11. ; two argument strings > (system* "/usr/local/bin/wget" "http://google.com; "-O /tmp/fooflaksdjdghk") /tmp/fooflaksdjdghk: No such file or directory #f ; one argument string > (system* "/usr/local/bin/wget" "http://google.com -O /tmp/fooflaksdjdghk") --13:47:10-- http://google.com%20-o%20/tmp/fooflaksdjdghk => `fooflaksdjdghk' Resolving google.com -o ... failed: nodename nor servname provided, or not known. #f Okay, it's transforming -O into -o. Let's try the long form: > (system* "/usr/local/bin/wget" "http://google.com; > "--output-document=/tmp/zot") ...lots of spew... #t Yep, that works. Okay, now let's try it with an actual filename, unescaped to start. I expect this to fail: > (system* "/usr/local/bin/wget" "http://google.com; > "--output-document=/tmp/Sara's birthday, 9/12/01-18/181") /tmp/Sara's birthday, 9/12/01-18/181: No such file or directory #f Sure enough. Oh good, we're back to shell-style escapes where all this started. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
David Storrs wrote on 12/28/18 11:09 PM: I am using 'system' to offload some work onto wget and other applications in a few one-off scripts. Is there an easy way to escape a string so it's suitable for usage in the shell? Things like backwhacking all the quotes and relevant spaces and such. As you know (just to articulate it for the list), processing command lines through the host shell is a little scary, and an imperfection here can be a huge security hole, or otherwise cause failures. We can try to do escaping/quoting, but there are so many potential special cases on different host systems. Fortunately, there might be an alternative... If you're getting a command line from elsewhere, safer is to parse it yourself into process arguments in Racket, and pass those to something like `system*`, so that you know it's done correctly, and there's no shell-specific special characters or bugs or Unicode tricks or anything else. There are cases in which you have a host shell command line that you can't parse it yourself (e.g., a process takes a command line as an argument, or you have to use a subshell command line). If you really have to do that, you could look at how I did it for a side project several years ago, but it isn't quite my current thinking: https://www.neilvandyke.org/racket/sudo/ (IIRC, I probably wrote that `sudo` module for a module possibly even more scary: https://www.neilvandyke.org/racket/parted/ ) -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Escaping strings for the shell
On Fri, 28 Dec 2018 at 23:09 -0500, David Storrs wrote: > I am using 'system' to offload some work onto wget and other > applications in a few one-off scripts. Is there an easy way to escape > a string so it's suitable for usage in the shell? Things like > backwhacking all the quotes and relevant spaces and such. Is avoiding the shell altogether an option? You could use "system*" -- Jack M. Rosenthal http://jack.rosenth.al /* Return code=1: generic error condition Return code=2: all other error conditions */ -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. signature.asc Description: PGP signature
