[
https://issues.apache.org/jira/browse/HADOOP-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18073508#comment-18073508
]
ASF GitHub Bot commented on HADOOP-1593:
----------------------------------------
steveloughran commented on code in PR #8400:
URL: https://github.com/apache/hadoop/pull/8400#discussion_r3080384818
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java:
##########
@@ -1184,7 +1209,7 @@ ReadBufferManager getReadBufferManager() {
*/
@Override
public int minSeekForVectorReads() {
Review Comment:
I'm actually going to recommend *not* coalescing vectors unless you have a
good strategy to deal with partial failures, memory releases, retries etc.
I did try to do that in s3a and gave up, even after extending the api to
allow a "release" operation to be passed (which parquet passes down FWIW).
We've never seen failures with s3 in production and coalesced ranges as
parquet/orc rowgroups are too far apart. So it's better to focus on retry and
recovery there than range coalescing
> FsShell should work with paths in non-default FileSystem
> --------------------------------------------------------
>
> Key: HADOOP-1593
> URL: https://issues.apache.org/jira/browse/HADOOP-1593
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Reporter: Doug Cutting
> Assignee: Mahadev Konar
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Attachments: patch_1593.patch, patch_1593_1.patch
>
>
> If the default filesystem is, e.g., hdfs://foo:8888/, one should still be
> able to do 'bin/hadoop fs -ls hdfs://bar:9999/' or 'bin/hadoop fs -ls
> s3://cutting/foo'. Currently these generate a filesystem mismatch exception.
> This is because FsShell assumes that all paths are in the default
> FileSystem. Rather, the default filesystem should only be used for paths
> that do not specify a FileSystem. This would easily be accomplished by using
> Path#getFileSystem().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
