Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
epugh commented on code in PR #2479: URL: https://github.com/apache/solr/pull/2479#discussion_r1774352198 ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -0,0 +1,167 @@ += Stream Tool +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +The Stream tool allows you to run a xref:streaming-expressions.adoc[] and see the results from the command line. +It is very similar to the xref:stream-screen.adoc[], but is part of the `bin/solr` CLI. + +To run it, open a window and enter: + +[,console] + +$ bin/solr stream --header -c techproducts --delimiter=\| 'search(techproducts,q="name:memory",fl="name,price")' + + +This will run the provided streaming expression on the `techproducts` collection and produce: + +[,console] + +name|price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail|185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail|74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM| + + +TIP: Notice how we used the pipe character (|) as the delimiter? It required a backslash for escaping it so it wouldn't be treated as a pipe with in the shell script. + +You can also specify a file with the suffix `.expr` containing your streaming expression. +This is useful for longer expressions or if you having command line parsing issues with your expression. + +Assuming you have create the file `stream.expr` with the contents: + + +# Stream a search + +search( + techproducts, + q="name:memory", + fl="name,price", + sort="price desc" +) + + +Then you can run it on the Solr collection `techproducts`, specifying you want a header row: + +[,console] + +$ bin/solr stream --header -c techproducts stream.expr + + +And this will produce: + +[,console] + +name price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail 185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail 74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM + + +The `--help` (or simply `-h`) option will output information on its usage (i.e., `bin/solr stream --help)`. + +== Using the bin/solr stream Tool + +To use the tool you need to provide the streaming expression either inline as the last argument, or provide a file ending in `.expr` that contains the expression. + +The basic usage of `bin/solr stream` is: + +[source,plain] + +usage: bin/solr stream [--array-delimiter ] [-c ] [--delimiter ] [-f ] [--header] + [-u ] [-url ] [--workers ] [-z ] + +List of options: +--array-delimiterThe delimiter multi-valued fields. (default=|) + -c,--nameName of the collection to execute on if workers are 'solr'. Required for 'solr' +worker. +--delimiter The output delimiter. (default=tab). + -f,--fieldsThe fields in the tuples to output. (defaults to fields in the first tuple of result +set). +--headerWhether or not to include a header line. (default=false) + -u,--credentials Credentials in the format username:password. Example: --credentials solr:SolrRocks + -url,--solr-url Base Solr URL, which can be used to determine the zk-host if that's not known; +defaults to: http://localhost:8983. + -e,--executionExecution context is either 'local' or 'solr'. Default is 'solr' + -z,--zk-host Zookeeper connection string; unnecessary if ZK_HOST is defined in solr.in.sh; +otherwise, defaults to localhost:9983. + + +== Examples Using bin/solr stream + +There are several ways to use `bin/solr stream`. +This section presents several examples. + +=== Executing Expression Locally + +Streaming Expressions by default are executed in the Solr cluster. +However there are use cases where you want to interact with data in your local environm
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
epugh commented on code in PR #2479: URL: https://github.com/apache/solr/pull/2479#discussion_r1773657326 ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -71,13 +72,15 @@ CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Me A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM -The `--help` (or simply `-h`) option will output information on its usage (i.e., `bin/solr stream --help)`. - == Using the bin/solr stream Tool +NOTE: The Stream Tool is classified as "experimental". Review Comment: love it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
gerlowskija commented on code in PR #2479: URL: https://github.com/apache/solr/pull/2479#discussion_r1773493224 ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -71,13 +72,15 @@ CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Me A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM -The `--help` (or simply `-h`) option will output information on its usage (i.e., `bin/solr stream --help)`. - == Using the bin/solr stream Tool +NOTE: The Stream Tool is classified as "experimental". Review Comment: Might be worth putting this at the top of the page. Seems a little "buried" as-is. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
gerlowskija commented on code in PR #2479: URL: https://github.com/apache/solr/pull/2479#discussion_r1773483777 ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -0,0 +1,167 @@ += Stream Tool +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +The Stream tool allows you to run a xref:streaming-expressions.adoc[] and see the results from the command line. +It is very similar to the xref:stream-screen.adoc[], but is part of the `bin/solr` CLI. + +To run it, open a window and enter: + +[,console] + +$ bin/solr stream --header -c techproducts --delimiter=\| 'search(techproducts,q="name:memory",fl="name,price")' + + +This will run the provided streaming expression on the `techproducts` collection and produce: + +[,console] + +name|price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail|185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail|74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM| + + +TIP: Notice how we used the pipe character (|) as the delimiter? It required a backslash for escaping it so it wouldn't be treated as a pipe with in the shell script. + +You can also specify a file with the suffix `.expr` containing your streaming expression. +This is useful for longer expressions or if you having command line parsing issues with your expression. + +Assuming you have create the file `stream.expr` with the contents: + + +# Stream a search + +search( + techproducts, + q="name:memory", + fl="name,price", + sort="price desc" +) + + +Then you can run it on the Solr collection `techproducts`, specifying you want a header row: + +[,console] + +$ bin/solr stream --header -c techproducts stream.expr + + +And this will produce: + +[,console] + +name price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail 185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail 74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM + + +The `--help` (or simply `-h`) option will output information on its usage (i.e., `bin/solr stream --help)`. + +== Using the bin/solr stream Tool + +To use the tool you need to provide the streaming expression either inline as the last argument, or provide a file ending in `.expr` that contains the expression. + +The basic usage of `bin/solr stream` is: + +[source,plain] + +usage: bin/solr stream [--array-delimiter ] [-c ] [--delimiter ] [-f ] [--header] + [-u ] [-url ] [--workers ] [-z ] + +List of options: +--array-delimiterThe delimiter multi-valued fields. (default=|) + -c,--nameName of the collection to execute on if workers are 'solr'. Required for 'solr' +worker. +--delimiter The output delimiter. (default=tab). + -f,--fieldsThe fields in the tuples to output. (defaults to fields in the first tuple of result +set). +--headerWhether or not to include a header line. (default=false) + -u,--credentials Credentials in the format username:password. Example: --credentials solr:SolrRocks + -url,--solr-url Base Solr URL, which can be used to determine the zk-host if that's not known; +defaults to: http://localhost:8983. + -e,--executionExecution context is either 'local' or 'solr'. Default is 'solr' Review Comment: I noodled on this a bit but still couldn't come up with anything I liked better, so maybe you picked the best option. +1 for the current enum values with the "(i.e. CLI process)" clarification you suggested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infra
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
epugh commented on code in PR #2479: URL: https://github.com/apache/solr/pull/2479#discussion_r1763521501 ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -0,0 +1,167 @@ += Stream Tool +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +The Stream tool allows you to run a xref:streaming-expressions.adoc[] and see the results from the command line. +It is very similar to the xref:stream-screen.adoc[], but is part of the `bin/solr` CLI. + +To run it, open a window and enter: + +[,console] + +$ bin/solr stream --header -c techproducts --delimiter=\| 'search(techproducts,q="name:memory",fl="name,price")' + + +This will run the provided streaming expression on the `techproducts` collection and produce: + +[,console] + +name|price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail|185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail|74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM| + + +TIP: Notice how we used the pipe character (|) as the delimiter? It required a backslash for escaping it so it wouldn't be treated as a pipe with in the shell script. + +You can also specify a file with the suffix `.expr` containing your streaming expression. +This is useful for longer expressions or if you having command line parsing issues with your expression. + +Assuming you have create the file `stream.expr` with the contents: + + +# Stream a search + +search( + techproducts, + q="name:memory", + fl="name,price", + sort="price desc" +) + + +Then you can run it on the Solr collection `techproducts`, specifying you want a header row: + +[,console] + +$ bin/solr stream --header -c techproducts stream.expr + + +And this will produce: + +[,console] + +name price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail 185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail 74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM + + +The `--help` (or simply `-h`) option will output information on its usage (i.e., `bin/solr stream --help)`. + +== Using the bin/solr stream Tool + +To use the tool you need to provide the streaming expression either inline as the last argument, or provide a file ending in `.expr` that contains the expression. + +The basic usage of `bin/solr stream` is: + +[source,plain] + +usage: bin/solr stream [--array-delimiter ] [-c ] [--delimiter ] [-f ] [--header] + [-u ] [-url ] [--workers ] [-z ] + +List of options: +--array-delimiterThe delimiter multi-valued fields. (default=|) + -c,--nameName of the collection to execute on if workers are 'solr'. Required for 'solr' +worker. +--delimiter The output delimiter. (default=tab). + -f,--fieldsThe fields in the tuples to output. (defaults to fields in the first tuple of result +set). +--headerWhether or not to include a header line. (default=false) + -u,--credentials Credentials in the format username:password. Example: --credentials solr:SolrRocks + -url,--solr-url Base Solr URL, which can be used to determine the zk-host if that's not known; +defaults to: http://localhost:8983. + -e,--executionExecution context is either 'local' or 'solr'. Default is 'solr' Review Comment: Thoughts on: ``` Execution context is either 'local' (i.e CLI process) or 'solr'. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsub
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
gerlowskija commented on code in PR #2479: URL: https://github.com/apache/solr/pull/2479#discussion_r1761633393 ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -0,0 +1,167 @@ += Stream Tool +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +The Stream tool allows you to run a xref:streaming-expressions.adoc[] and see the results from the command line. +It is very similar to the xref:stream-screen.adoc[], but is part of the `bin/solr` CLI. + +To run it, open a window and enter: + +[,console] + +$ bin/solr stream --header -c techproducts --delimiter=\| 'search(techproducts,q="name:memory",fl="name,price")' + + +This will run the provided streaming expression on the `techproducts` collection and produce: + +[,console] + +name|price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail|185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail|74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM| + + +TIP: Notice how we used the pipe character (|) as the delimiter? It required a backslash for escaping it so it wouldn't be treated as a pipe with in the shell script. + +You can also specify a file with the suffix `.expr` containing your streaming expression. +This is useful for longer expressions or if you having command line parsing issues with your expression. + +Assuming you have create the file `stream.expr` with the contents: + + +# Stream a search + +search( + techproducts, + q="name:memory", + fl="name,price", + sort="price desc" +) + + +Then you can run it on the Solr collection `techproducts`, specifying you want a header row: + +[,console] + +$ bin/solr stream --header -c techproducts stream.expr + + +And this will produce: + +[,console] + +name price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail 185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail 74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM + + +The `--help` (or simply `-h`) option will output information on its usage (i.e., `bin/solr stream --help)`. + +== Using the bin/solr stream Tool + +To use the tool you need to provide the streaming expression either inline as the last argument, or provide a file ending in `.expr` that contains the expression. + +The basic usage of `bin/solr stream` is: + +[source,plain] + +usage: bin/solr stream [--array-delimiter ] [-c ] [--delimiter ] [-f ] [--header] + [-u ] [-url ] [--workers ] [-z ] + +List of options: +--array-delimiterThe delimiter multi-valued fields. (default=|) + -c,--nameName of the collection to execute on if workers are 'solr'. Required for 'solr' +worker. +--delimiter The output delimiter. (default=tab). + -f,--fieldsThe fields in the tuples to output. (defaults to fields in the first tuple of result +set). +--headerWhether or not to include a header line. (default=false) + -u,--credentials Credentials in the format username:password. Example: --credentials solr:SolrRocks + -url,--solr-url Base Solr URL, which can be used to determine the zk-host if that's not known; +defaults to: http://localhost:8983. + -e,--executionExecution context is either 'local' or 'solr'. Default is 'solr' Review Comment: [0] I don't love this terminology, but I don't have anything better in mind (yet). "Local" to me could be misconstrued by folks running 'bin/solr' on a box that also happens to have Solr running. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
gerlowskija commented on code in PR #2479: URL: https://github.com/apache/solr/pull/2479#discussion_r1760947380 ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -0,0 +1,167 @@ += Stream Tool +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +The Stream tool allows you to run a xref:streaming-expressions.adoc[] and see the results from the command line. +It is very similar to the xref:stream-screen.adoc[], but is part of the `bin/solr` CLI. Review Comment: [0] Might be worth mentioning here the other differentiator - that this executes some streams "locally"? ## solr/solr-ref-guide/modules/query-guide/pages/stream-tool.adoc: ## @@ -0,0 +1,167 @@ += Stream Tool +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +The Stream tool allows you to run a xref:streaming-expressions.adoc[] and see the results from the command line. +It is very similar to the xref:stream-screen.adoc[], but is part of the `bin/solr` CLI. + +To run it, open a window and enter: + +[,console] + +$ bin/solr stream --header -c techproducts --delimiter=\| 'search(techproducts,q="name:memory",fl="name,price")' + + +This will run the provided streaming expression on the `techproducts` collection and produce: + +[,console] + +name|price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail|185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail|74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM| + + +TIP: Notice how we used the pipe character (|) as the delimiter? It required a backslash for escaping it so it wouldn't be treated as a pipe with in the shell script. + +You can also specify a file with the suffix `.expr` containing your streaming expression. +This is useful for longer expressions or if you having command line parsing issues with your expression. + +Assuming you have create the file `stream.expr` with the contents: + + +# Stream a search + +search( + techproducts, + q="name:memory", + fl="name,price", + sort="price desc" +) + + +Then you can run it on the Solr collection `techproducts`, specifying you want a header row: + +[,console] + +$ bin/solr stream --header -c techproducts stream.expr + + +And this will produce: + +[,console] + +name price +CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail 185.0 +CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail 74.99 +A-DATA V-Series 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - OEM + + +The `--help` (or simply `-h`) option will output information on its usage (i.e., `bin/solr stream --help)`. + +== Using the bin/solr stream Tool + +To use the tool you need to provide the streaming expression either inline as the last argument, or provide a file ending in `.expr` that contains the expression. + +The basic usage of `bin/solr stream` is: + +[source,plain] + +usage: bin/solr stream [--array-delimiter ] [-c ] [--delimiter ] [-f ] [--header] + [-u ] [-url ] [--workers ] [-z ] + +List of options: +--array-delimiterThe delimiter multi-valued fields. (default=|) + -c,--nameName of the collection to execute on if workers are 'solr'. Required
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
epugh commented on PR #2479: URL: https://github.com/apache/solr/pull/2479#issuecomment-2282307377 @gerlowskija since you provided some early review, do you think the docs I've added etc are enough that I can merge this in? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
epugh commented on PR #2479: URL: https://github.com/apache/solr/pull/2479#issuecomment-2246259400 Okay, I think this is ready for review! I've added some docs.. I especially liked being able to cat some local data right into a Solr collection! ``` cat example/exampledocs/books.csv | bin/solr stream -e local 'update(gettingstarted,parseCSV(stdin()))' ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
epugh commented on PR #2479: URL: https://github.com/apache/solr/pull/2479#issuecomment-2149443128 Thanks for sharing the feedback @gerlowskija ! I think the value of the tool is only there if your second comment about being able to run a streaming expression locally is valid, and then having it do what yoru first comment highlights falls out easy, otherwise it really is a thin wrapper/duplication of the `bin/solr api` call. Especially without any special value add in formatting tuples or error handling etc. I do believe the second part is the really cool thing, that I can run a streaming expression locally and use it to process some data. We clearly need some way of specifying where the processing is happening, in the cluster or locally. I was trying to think if we have any other places in Solr where we define "Where am I doing work" that might provide a name for a parameter. `bin/solr stream --environment cluster BLAH` ? The `search()` expression has a `qt` parameter.. `bin/solr stream -qt=/stream BLAH` ? Reading through docs more, we have the `parralel()` and it refers to workers.Maybe the command should be something like `bin/solr stream --workers=local BLAH` which would run on your laptop, and if you don't specify --workers then it runs on the cluster via `/stream`? I have found that lots of streaming expressions don't require a Solr connection, especially during development. I'm just iterating on the logic, and I'm starting and ending iwth tuples.. it's only later when I get the mappings etc working that I then move to adding in my `search()` or `update()` clauses. Also, as far as docs go, we have a LONG way to go in Streaming expressions. It's both the best docuemnted code, with all the howtos and guides, but also, I find a million expressions that exist but don't show up in our reference docs ;-). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
Re: [PR] SOLR-14673: Add bin/solr stream CLI [solr]
gerlowskija commented on PR #2479: URL: https://github.com/apache/solr/pull/2479#issuecomment-2145825393 A few high-level questions/concerns: 1. `bin/solr` already has an "api" tool, which can be used to invoke streaming expressions e.g. `bin/solr api -get "$SOLR_URL/techproducts/stream?expr=search(techproducts)"`. I'm all for syntactic-sugar, but I wonder whether this is worth the maintenance cost if the main thing that it "buys" us is saving people from having to provide the full API path as the "api" tool requires? 2. If I'm reading the PR correctly, it looks like one other capability of the proposed `bin/solr stream` tool is that it can evaluate streams "locally" in some cases i.e. without a full running Solr. Which is pretty cool - you could imagine a real super-user doing some pretty involved ETL that builds off of an expression like: `update(techproducts, unique(cat(...)))`. But I'd worry about some of the documentation challenges surrounding this. For instance, how would a user know which expressions can be run locally, and which require a Solr to execute on? For expressions that have a mix of both locally and remotely-executed clauses, is there any way for a user to know which clauses are executed where? To clarify - I think the upside here is pretty cool, I'm just worried about the documentation end and what we might need to make it usable by folks in practice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[PR] SOLR-14673: Add bin/solr stream CLI [solr]
epugh opened a new pull request, #2479: URL: https://github.com/apache/solr/pull/2479 https://issues.apache.org/jira/browse/SOLR-14673 # Description Bring in code that @joel-bernstein wrote, but using the SolrCLI infrastructure. # Solution Another CLI client ;-) # Tests Copied over the basic tests from the patch. I still need to write an integration style test and ideally one that exercies the basic auth. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/solr/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [ ] I have developed this patch against the `main` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
