On Tue, Sep 19, 2023 at 09:53:25PM -0400, Karl Vogel wrote: > On Tue, Sep 19, 2023 at 10:42:14AM -0400, steve wrote: > > I have a list of 200 keywords and would like for every one to launch a > > search on a specific website and put the result(s) in a file [...] > > Take a list of words and turn it into a single Google query matching any > of them: > > me% cat keywords > corsairs > buccaneers > privateers > > Desired query: > > https://www.google.com/search?q=corsairs+OR+buccaneers+OR+privateers
It sounds more like steve wanted 200 separate searches, one for each keyword. For that, you'd use a very simple loop. > Script: > > me% cat search > #!/bin/sh > export PATH=/usr/local/bin:/bin:/usr/bin > set -o nounset > umask 022 > > query=$(tr "[:space:]" "+" < keywords | > sed -e 's/ /+OR+/g' -e 's/+$//' -e 's/+/+OR+/g') > > curl -s -L -o pirate.htm "https://www.google.com/search?q=${query}"; > ls -l pirate.htm > exit 0 Joining a list (array) of strings into a big string is so horribly ugly in /bin/sh. Switching to bash buys you so many better options. #!/bin/bash mapfile -t words < keywords printf -v query '%s+OR+' "${words[@]}" query=${query%+OR+} curl -s -L -o output.html "https://www.google.com/search?q=$query"; Of course, if you were writing this for publication, you'd want to URL-encode the keywords themselves, because they might contain spaces, plus signs, ampersands, etc.
