[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927886#comment-17927886
]
ASF subversion and git services commented on SOLR-17351:
Commit c5ce8dd3437210c6866bc6b197afb998047cc3b2 in solr's branch
refs/heads/branch_9x from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=c5ce8dd3437 ]
SOLR-17351: Decompose filestore "get file" API
This commit splits up the "get file" endpoint into a number of different
APIs. Specifically:
- metadata-fetching has been moved out to the endpoint,
GET/api/cluster/filestore/metadata/some/path.txt
- Filestore commands such as pushing/pulling files are now
available at: POST /api/cluster/filestore/commands
These divisions allow us to generate SolrRequest/SolrResponse classes
representing these APIs, meaning that SolrJ users no longer need to use
GenericSolrRequest/GenericSolrResponse.
(As a 9.x backport this commit retains the original form of these APIs
to retain backwards compatibility, but this support should be removed in
10.0)
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Attachments: SOLR-17351.test-failures.tgz
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927558#comment-17927558
]
ASF subversion and git services commented on SOLR-17351:
Commit 1f67b4f487cc371913fef581c59f4b44d3b3e2b5 in solr's branch
refs/heads/main from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=1f67b4f487c ]
SOLR-17351: Decompose filestore "get file" API (#3047)
This PR splits up the "get file" endpoint into a number of different APIs.
Specifically:
- metadata-fetching has been moved out to the endpoint,
GET/api/cluster/filestore/metadata/some/path.txt
- Filestore commands such as pushing/pulling files are now available at:
POST /api/cluster/filestore/commands
- Support for "JSON-ified" file data has been dropped in this PR (but will be
retained but deprecated in the eventual 9.x backport)
These divisions allow us to generate SolrRequest/SolrResponse classes
representing these APIs, meaning that SolrJ users no longer need to use
GenericSolrRequest/GenericSolrResponse.
(This commit apes an earlier commit which offered similar functionality
but caused a few test failures. These have now been fixed.)
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Attachments: SOLR-17351.test-failures.tgz
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927265#comment-17927265
]
Jason Gerlowski commented on SOLR-17351:
Yep, understood!
The revert was definitely my choice. I opted to be "trigger-happy" here
because (1) the PR definitely touches ObjectReleaseTraacker stuff, so the
failure makes sense even though I can't reproduce, and (2) this ticket isn't my
highest priority at the moment and I didn't want to leave anything potentially
broken while I scrounged up time to dig in.
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927243#comment-17927243
]
Chris M. Hostetter commented on SOLR-17351:
---
FWIW: I wasn't request you revert, just mentioning that it seemed to cause
failures that were suspiciously related and reverting locally made the failures
stop : )
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927238#comment-17927238
]
Jason Gerlowski commented on SOLR-17351:
Wow - commit to revert-request in under 2 hours - amazing! 😛
I'm not seeing those test failures locally, and they didn't fail on the PR
checks, but the stacktrace should give me all I need to investigate even if I
can't reproduce. Thanks for the thorough report! Reverting now...
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927241#comment-17927241
]
ASF subversion and git services commented on SOLR-17351:
Commit dd6e18e54ed32bb66f2552a7ff5bff23d3dc9834 in solr's branch
refs/heads/main from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=dd6e18e54ed ]
Revert "SOLR-17351: Decompose filestore "get file" API (#3047)"
This reverts commit 2ab12431c04a3d860d21d10cf45d2434ea50bbe8.
Hoss reported a test failure due to some ObjectReleaseTracker
violations. The InputStream handling did evolve a bit in the course of
review, so its possible that caused some issue. Undoing this change
while I investigate...
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927236#comment-17927236
]
Chris M. Hostetter commented on SOLR-17351:
---
Example...
{noformat}
> java.lang.AssertionError: ObjectTracker found 6 object(s) that were
not released!!! [Input, Input, Input, Input, Input, Input]
>
org.eclipse.jetty.client.util.InputStreamResponseListener$Input:org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException:
org.eclipse.jetty.client.util.InputStreamResponseListener$Input
>at
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:54)
>at
org.apache.solr.client.solrj.impl.Http2SolrClient$InputStreamReleaseTrackingResponseListener$ObjectReleaseTrackedInputStream.(Http2SolrClient.java:1213)
>at
org.apache.solr.client.solrj.impl.Http2SolrClient$InputStreamReleaseTrackingResponseListener.getInputStream(Http2SolrClient.java:1207)
>at
org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:519)
>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:260)
>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:276)
>at
org.apache.solr.filestore.DistribFileStore$FileInfo.fetchFileFromNodeAndPersist(DistribFileStore.java:192)
>at
org.apache.solr.filestore.DistribFileStore.fetch(DistribFileStore.java:437)
>at
org.apache.solr.filestore.ClusterFileStore.lambda$pullFileFromNode$5(ClusterFileStore.java:352)
>at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212)
>at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
>at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
>at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:380)
>at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
>at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
>at java.base/java.lang.Thread.run(Thread.java:1583)
>
>
org.eclipse.jetty.client.util.InputStreamResponseListener$Input:org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException:
org.eclipse.jetty.client.util.InputStreamResponseListener$Input
>at
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:54)
>at
org.apache.solr.client.solrj.impl.Http2SolrClient$InputStreamReleaseTrackingResponseListener$ObjectReleaseTrackedInputStream.(Http2SolrClient.java:1213)
>at
org.apache.solr.client.solrj.impl.Http2SolrClient$InputStreamReleaseTrackingResponseListener.getInputStream(Http2SolrClient.java:1207)
>at
org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:519)
>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:260)
>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:276)
>at
org.apache.solr.filestore.DistribFileStore$FileInfo.fetchFileFromNodeAndPersist(DistribFileStore.java:192)
>at
org.apache.solr.filestore.DistribFileStore.fetch(DistribFileStore.java:437)
>at
org.apache.solr.filestore.ClusterFileStore.lambda$pullFileFromNode$5(ClusterFileStore.java:352)
>at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212)
>at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
>at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
>at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:380)
>at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
>at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
>at java.base/java.lang.Thread.run(Thread.java:1583)
...
{noformat}
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore AP
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927232#comment-17927232
]
Chris M. Hostetter commented on SOLR-17351:
---
I'm seeing several test failures on main (at least one of which semes to
reproduce regardless of seed) that seem directly related to the
{{InputStreamResponse}} changes in this commit (removing the call to
{{ObjectReleaseTracker.release(inputStream)}}...
{noformat}
ERROR: The following test(s) have failed:
- org.apache.solr.cli.PackageToolTest (:solr:core)
Test history:
https://develocity.apache.org/scans/tests?search.rootProjectNames=solr-root&tests.container=org.apache.solr.cli.PackageToolTest
Test output:
/home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cli.PackageToolTest.txt
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.cli.PackageToolTest" "-Ptests.jvmargs=-XX:TieredStopAtLevel=1
-XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=CC0E7E60A363C961 -Ptests.file.encoding=ISO-8859-1
- org.apache.solr.pkg.TestPackages (:solr:core)
Test history:
https://develocity.apache.org/scans/tests?search.rootProjectNames=solr-root&tests.container=org.apache.solr.pkg.TestPackages
Test output:
/home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.pkg.TestPackages.txt
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.pkg.TestPackages" "-Ptests.jvmargs=-XX:TieredStopAtLevel=1
-XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=CC0E7E60A363C961 -Ptests.file.encoding=ISO-8859-1
- org.apache.solr.filestore.TestDistribFileStore (:solr:core)
Test history:
https://develocity.apache.org/scans/tests?search.rootProjectNames=solr-root&tests.container=org.apache.solr.filestore.TestDistribFileStore
Test output:
/home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.filestore.TestDistribFileStore.txt
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.filestore.TestDistribFileStore"
"-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
-XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=CC0E7E60A363C961 -Ptests.file.encoding=ISO-8859-1
- org.apache.solr.handler.TestContainerPlugin (:solr:core)
Test history:
https://develocity.apache.org/scans/tests?search.rootProjectNames=solr-root&tests.container=org.apache.solr.handler.TestContainerPlugin
Test output:
/home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.TestContainerPlugin.txt
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.handler.TestContainerPlugin"
"-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
-XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=CC0E7E60A363C961 -Ptests.file.encoding=ISO-8859-1
- org.apache.solr.pkg.PackageStoreSchemaPluginsTest (:solr:core)
Test history:
https://develocity.apache.org/scans/tests?search.rootProjectNames=solr-root&tests.container=org.apache.solr.pkg.PackageStoreSchemaPluginsTest
Test output:
/home/hossman/lucene/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.pkg.PackageStoreSchemaPluginsTest.txt
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.pkg.PackageStoreSchemaPluginsTest"
"-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
-XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=CC0E7E60A363C961 -Ptests.file.encoding=ISO-8859-1
{noformat}
reverting 2ab12431c04 locally seems to make these pass again.
Â
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927176#comment-17927176
]
ASF subversion and git services commented on SOLR-17351:
Commit 2ab12431c04a3d860d21d10cf45d2434ea50bbe8 in solr's branch
refs/heads/main from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=2ab12431c04 ]
SOLR-17351: Decompose filestore "get file" API (#3047)
This PR splits up the "get file" endpoint into a number of different APIs.
Specifically:
- metadata-fetching has been moved out to the endpoint,
GET/api/cluster/filestore/metadata/some/path.txt
- Filestore commands such as pushing/pulling files are now available at:
POST /api/cluster/filestore/commands
- Support for "JSON-ified" file data has been dropped in this PR (but will be
retained but deprecated in the eventual 9.x backport)
These divisions allow us to generate SolrRequest/SolrResponse classes
representing these APIs, meaning that SolrJ users no longer need to use
GenericSolrRequest/GenericSolrResponse.
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910654#comment-17910654
]
David Smiley commented on SOLR-17351:
-
Okay; I get it.
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910615#comment-17910615
]
Jason Gerlowski commented on SOLR-17351:
I guess my concern with dropping the "filestore" segment is that
/api/cluster/files could, in the head of a newer user, refer to any number of
things. The API could ostensibly allow access to files in SOLR_HOME, or to
configset files, or... The "filestore" path segment makes the API intention
hard to misconstrue.
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17909919#comment-17909919
]
David Smiley commented on SOLR-17351:
-
Overall I like this JIRA. Some minor feedback:
* The paths are rather long. Assuming getting a file is common, it'd be nice
to support /api/cluster/files/some/path.txt (omitting filestore segment that
you proposed). If that seems agreeable but "what about the others", they could
be as you propose or do these translations: "filestore/metadata" ->
"files_metadata", etc.
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17909741#comment-17909741
]
Jason Gerlowski commented on SOLR-17351:
Started thinking about this again after some [recent discussion in
Slack|https://the-asf.slack.com/archives/CEKUCUNE9/p1735936717913429?thread_ts=1735927175.702769&cid=CEKUCUNE9].
I proposed above that we take the existing API and break it up into multiple
endpoints. Here's one option for how that could be done, that could be done in
a backcompat-friendly way without too much effort. At a high-level, the idea
is to introduce a "filestore" path segment, which can hold different sub paths
underneath to support the different ops:
# Tweak existing filestore APIs to take an (optional in 9.x) "filestore" path
segment. e.g. Delete-file would become {{DELETE
/api/cluster/filestore/files/some/path.txt}}. (This could be a simple tweak in
the '@Path' JAX-RS annotation for these methods)
# Add a "get file content" API at {{GET
/api/cluster/filestore/files/some/path.txt}}.
## This would only support fetching raw file contents. The special JSON-ified
response with wt=json would not be available at this new path.
# Add a "get file metadata" API at {{GET
/api/cluster/filestore/metadata/some/path.txt}}
# Add a "filestore commands" API for syncing and pulling file contents at
{{POST /api/cluster/filestore/commands?}}
Let me know what you guys think; I can try to put together a draft of this soon
if I find time.
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860649#comment-17860649
]
Eric Pugh commented on SOLR-17351:
--
Fair point.  I've never done the cluster upgrade in the rolling pattern..Â
I've only ever done blue/green, so I keep forgetting about that aspect!
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860617#comment-17860617
]
Jason Gerlowski commented on SOLR-17351:
The main reason to defer updating Solr's internal usage in 9.x is that it risks
breaking users in the midst of a minor version upgrade. e.g. Imagine you're
doing a rolling-upgrade from 9.5 to 9.8, with nodes on a mixture of those two
versions. If a 9.8 node makes an internal filestore API call to another part
of the cluster, it can't use the "new" version of the API because the request
might be going to a node that's still on 9.5 (which won't have the "new" API
yet).
I'm tempted to say: "Why don't we just document that users should avoid
package-manager/file-store operations during an upgrade?"Â But even that
wouldn't necessarily solve the problem, because I imagine Solr does a bunch of
package-manager/file-store stuff internally on startup (e.g. so that it can
fetch any jars, etc. that it's supposed to have on the classpath). So even if
users don't make any explicit calls to the package-manager, upgrades will still
be broken and messy if our internal usage starts assuming an API that might not
be available on all nodes in the cluster.
If there's some way around that, I'd love to be more aggressive and change the
internal-usage for 9.x. But I can't see a way around it. Maybe someone with
more package-store knowledge could correct or confirm my suspicions here.Â
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860598#comment-17860598
]
Eric Pugh commented on SOLR-17351:
--
I like your plan, however I'd prefer to see the internal usage of these APIs to
happen in 9.x as well. My theory is that this is the best way to confirm that
the new API meets our needs, otherwise we are building it in a vacuum. Plus,
this ensures the new API works as well as the old one since Solr is a key user
of these APIs!
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17351) Cosmetic changes to v2 filestore "get file" API
[
https://issues.apache.org/jira/browse/SOLR-17351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860511#comment-17860511
]
Jason Gerlowski commented on SOLR-17351:
The big question, before we change this API, is whether the change needs to be
backwards compatible.
Technically at least, I think the answer is "no". All of our v2 APIs,
including this filestore API, are currently marked as experimental, which was
done intentionally to enable us to evolve them more freely. But these
filestore APIs seem like a bit of an exception in that they have no v1
equivalent. It's one thing to treat an API as experimental when there's a more
stable alternative to point users towards. But without that, making breaking
changes feels a little inconsiderate.
My vote is that we make at least some concession to backcompat:
 - introduce updated, more REST-ful versions of these APIs in 9.7, alongside
the versions that exist today
 - remove the older versions of the API in 10.0
 - Solr's internal usage of these APIs would continue to use the older
versions in 9.x releases, but switch to using the newer versions with 10.0
Any objections to that approach?
> Cosmetic changes to v2 filestore "get file" API
> ---
>
> Key: SOLR-17351
> URL: https://issues.apache.org/jira/browse/SOLR-17351
> Project: Solr
> Issue Type: Sub-task
> Components: Package Manager, v2 API
>Affects Versions: 9.6.1
>Reporter: Jason Gerlowski
>Priority: Minor
>
> Solr's filestore APIs fit well with the REST-ful design we're targeting with
> our v2 APIs, with one large exception: the "get file" API current available
> at {{GET /api/node/files/somePath.txt}}. This API stands out for a few
> reasons:
> 1. It uses a different path-prefix than all other filestore APIs. (i.e.
> {{/api/node/files}} instead of {{/api/cluster/files}})
> 2. It exposes 4 or 5 conceptually distinct operations. Obviously in the
> "default" case it allows callers to retrieve filestore contents, but based on
> query params it can instead:
> - return filestore entry metadata (when {{meta=true}} is specified)
> - instruct the receiving Solr node to pull a file from another node's
> filestore and cache it locally (when {{getFrom=someOtherNode}} is specified)
> - instruct the receiving Solr node to push its cached copy of a file out to
> all other Solr nodes (when {{sync=true}} is specified)
> 3. Even in the default case of returning "raw" filestore contents, the API
> can provide two different styles of response:
> - if {{wt=json}} is specified Solr will take the filestore entry bytes,
> attempt to stringify them, and then return a JSON object that uses this
> string as the value for a "response" key. It's unclear how this would work
> for binary content
> - for all other values of "wt", the API will return the raw file content.
> We should reconsider this endpoint and see if it can't be massaged into being
> more in line with our other v2 APIs. Some cosmetic tweaks will go a long
> way, but the biggest benefit is likely to come from breaking the endpoint up
> into multiple distinct APIs. In its current form, the API returns such a
> variety of responses that it's hard to write client code for. (I suspect
> this is the main reason these "filestore" APIs were never made available in
> SolrJ.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
