[jira] [Commented] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194589#comment-17194589 ] zhenzhao wang commented on HADOOP-15891: Thanks a lot! [~umamaheswararao] > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: viewfs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: HADOOP-15891.015.patch, HDFS-13948.001.patch, > HDFS-13948.002.patch, HDFS-13948.003.patch, HDFS-13948.004.patch, > HDFS-13948.005.patch, HDFS-13948.006.patch, HDFS-13948.007.patch, > HDFS-13948.008.patch, HDFS-13948.009.patch, HDFS-13948.011.patch, > HDFS-13948.012.patch, HDFS-13948.013.patch, HDFS-13948.014.patch, HDFS-13948_ > Regex Link Type In Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount > Table-v1.pdf > > Time Spent: 9h > Remaining Estimate: 0h > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17247) Support Non-Path Based FileSystem API in Regex Mount Points
[ https://issues.apache.org/jira/browse/HADOOP-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-17247: --- Target Version/s: 3.4.0 (was: 3.3.1) > Support Non-Path Based FileSystem API in Regex Mount Points > --- > > Key: HADOOP-17247 > URL: https://issues.apache.org/jira/browse/HADOOP-17247 > Project: Hadoop Common > Issue Type: Sub-task > Components: viewfs >Reporter: zhenzhao wang >Priority: Major > > Regex mount points create ChoRootedFileSystem while accessing. And we > couldn't know the underlying filesystems ahead. This won't work with non-path > based APIs such as getAdditionalTokenIssuers. Instead of totaly unsuport it, > we should support APIs to some extend. The could be done recording the > FileSystem created and perform APIs for FileSystem instances created for > ViewFileSystem. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17247) Support Non-Path Based FileSystem API in Regex Mount Points
[ https://issues.apache.org/jira/browse/HADOOP-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192645#comment-17192645 ] zhenzhao wang commented on HADOOP-17247: [~ste...@apache.org] Thanks, the parent patch is not merged yet. So I left the affected version. As for target version, I chose the next unreleased version, let me know whether it makes sense to you. > Support Non-Path Based FileSystem API in Regex Mount Points > --- > > Key: HADOOP-17247 > URL: https://issues.apache.org/jira/browse/HADOOP-17247 > Project: Hadoop Common > Issue Type: Sub-task > Components: viewfs >Reporter: zhenzhao wang >Priority: Major > > Regex mount points create ChoRootedFileSystem while accessing. And we > couldn't know the underlying filesystems ahead. This won't work with non-path > based APIs such as getAdditionalTokenIssuers. Instead of totaly unsuport it, > we should support APIs to some extend. The could be done recording the > FileSystem created and perform APIs for FileSystem instances created for > ViewFileSystem. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17247) Support Non-Path Based FileSystem API in Regex Mount Points
[ https://issues.apache.org/jira/browse/HADOOP-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-17247: --- Target Version/s: 3.3.1 > Support Non-Path Based FileSystem API in Regex Mount Points > --- > > Key: HADOOP-17247 > URL: https://issues.apache.org/jira/browse/HADOOP-17247 > Project: Hadoop Common > Issue Type: Sub-task > Components: viewfs >Reporter: zhenzhao wang >Priority: Major > > Regex mount points create ChoRootedFileSystem while accessing. And we > couldn't know the underlying filesystems ahead. This won't work with non-path > based APIs such as getAdditionalTokenIssuers. Instead of totaly unsuport it, > we should support APIs to some extend. The could be done recording the > FileSystem created and perform APIs for FileSystem instances created for > ViewFileSystem. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17247) Support Non-Path Based FileSystem API in Regex Mount Points
[ https://issues.apache.org/jira/browse/HADOOP-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-17247: --- Component/s: viewfs > Support Non-Path Based FileSystem API in Regex Mount Points > --- > > Key: HADOOP-17247 > URL: https://issues.apache.org/jira/browse/HADOOP-17247 > Project: Hadoop Common > Issue Type: Sub-task > Components: viewfs >Reporter: zhenzhao wang >Priority: Major > > Regex mount points create ChoRootedFileSystem while accessing. And we > couldn't know the underlying filesystems ahead. This won't work with non-path > based APIs such as getAdditionalTokenIssuers. Instead of totaly unsuport it, > we should support APIs to some extend. The could be done recording the > FileSystem created and perform APIs for FileSystem instances created for > ViewFileSystem. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17248) Support ChRootedFileSystem level cache for Regex Mount points
zhenzhao wang created HADOOP-17248: -- Summary: Support ChRootedFileSystem level cache for Regex Mount points Key: HADOOP-17248 URL: https://issues.apache.org/jira/browse/HADOOP-17248 Project: Hadoop Common Issue Type: Sub-task Reporter: zhenzhao wang Support ChRootedFileSystem level cache for Regex Mount points, so users don't need default change rename strategy settings to use rename API. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17247) Support Non-Path Based FileSystem API in Regex Mount Points
zhenzhao wang created HADOOP-17247: -- Summary: Support Non-Path Based FileSystem API in Regex Mount Points Key: HADOOP-17247 URL: https://issues.apache.org/jira/browse/HADOOP-17247 Project: Hadoop Common Issue Type: Sub-task Reporter: zhenzhao wang Regex mount points create ChoRootedFileSystem while accessing. And we couldn't know the underlying filesystems ahead. This won't work with non-path based APIs such as getAdditionalTokenIssuers. Instead of totaly unsuport it, we should support APIs to some extend. The could be done recording the FileSystem created and perform APIs for FileSystem instances created for ViewFileSystem. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17243) Add FileContext based Regex Mountpoint implementation.
[ https://issues.apache.org/jira/browse/HADOOP-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-17243: --- Parent: HADOOP-15891 Issue Type: Sub-task (was: Task) > Add FileContext based Regex Mountpoint implementation. > -- > > Key: HADOOP-17243 > URL: https://issues.apache.org/jira/browse/HADOOP-17243 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: zhenzhao wang >Priority: Major > > This Jira is created to track tasks for FileContext based Regex Mountpoint > implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17243) Add FileContext based Regex Mountpoint implementation.
zhenzhao wang created HADOOP-17243: -- Summary: Add FileContext based Regex Mountpoint implementation. Key: HADOOP-17243 URL: https://issues.apache.org/jira/browse/HADOOP-17243 Project: Hadoop Common Issue Type: Task Reporter: zhenzhao wang This Jira is created to track tasks for FileContext based Regex Mountpoint implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17238) Add ViewFileSystem/InodeTree Mount points Resolution Cache
zhenzhao wang created HADOOP-17238: -- Summary: Add ViewFileSystem/InodeTree Mount points Resolution Cache Key: HADOOP-17238 URL: https://issues.apache.org/jira/browse/HADOOP-17238 Project: Hadoop Common Issue Type: Improvement Components: viewfs Reporter: zhenzhao wang Assignee: zhenzhao wang Now the ViewFileSystem walkthrough InodeTree to resolve the passed-in path. This could be optimized by adding a cache of resolution result. It will benefit all kinds of mount points. And it is also very useful to reduce the Regex parse cost of Regex Mount points introduced in HADOOP-15891 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17068) client fails forever when namenode ipaddr changed
[
https://issues.apache.org/jira/browse/HADOOP-17068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173544#comment-17173544
]
zhenzhao wang commented on HADOOP-17068:
We had seen the problem multiple times too. One workaround we had been using
for years is to increase dfs.client.failover.connection.retries.on.timeouts to
3. It will help in the previous HDFS client version.
> client fails forever when namenode ipaddr changed
> -
>
> Key: HADOOP-17068
> URL: https://issues.apache.org/jira/browse/HADOOP-17068
> Project: Hadoop Common
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Sean Chow
>Assignee: Sean Chow
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-17068.001.patch, HDFS-15390.01.patch
>
>
> For machine replacement, I replace my standby namenode with a new ipaddr and
> keep the same hostname. Also update the client's hosts to make it resolve
> correctly
> When I try to run failover to transite the new namenode(let's say nn2), the
> client will fail to read or write forever until it's restarted.
> That make yarn nodemanager in sick state. Even the new tasks will encounter
> this exception too. Until all nodemanager restart.
>
> {code:java}
> 20/06/02 15:12:25 WARN ipc.Client: Address change detected. Old:
> nn2-192-168-1-100/192.168.1.100:9000 New: nn2-192-168-1-100/192.168.1.200:9000
> 20/06/02 15:12:25 DEBUG ipc.Client: closing ipc connection to
> nn2-192-168-1-100/192.168.1.200:9000: Connection refused
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:608)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:707)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:368)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1517)
> at org.apache.hadoop.ipc.Client.call(Client.java:1440)
> at org.apache.hadoop.ipc.Client.call(Client.java:1401)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
> at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:399)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:193)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> {code}
>
> We can see the client has {{Address change detected}}, but it still fails. I
> find out that's because when method {{updateAddress()}} return true, the
> {{handleConnectionFailure()}} thow an exception that break the next retry
> with the right ipaddr.
> Client.java: setupConnection()
> {code:java}
> } catch (ConnectTimeoutException toe) {
> /* Check for an address change and update the local reference.
>* Reset the failure counter if the address was changed
>*/
> if (updateAddress()) {
> timeoutFailures = ioFailures = 0;
> }
> handleConnectionTimeout(timeoutFailures++,
> maxRetriesOnSocketTimeouts, toe);
> } catch (IOException ie) {
> if (updateAddress()) {
> timeoutFailures = ioFailures = 0;
> }
> // because the namenode ip changed in updateAddress(), the old namenode
> ipaddress cannot be accessed now
> // handleConnectionFailure will thow an exception, the next retry never have
> a chance to use the right server updated in updateAddress()
> handleConnectionFailure(ioFailures++, ie);
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169709#comment-17169709 ] zhenzhao wang commented on HADOOP-15891: Updated pull request as the old one was close and I have no perm to reopen it. > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HADOOP-15891.015.patch, HDFS-13948.001.patch, > HDFS-13948.002.patch, HDFS-13948.003.patch, HDFS-13948.004.patch, > HDFS-13948.005.patch, HDFS-13948.006.patch, HDFS-13948.007.patch, > HDFS-13948.008.patch, HDFS-13948.009.patch, HDFS-13948.011.patch, > HDFS-13948.012.patch, HDFS-13948.013.patch, HDFS-13948.014.patch, HDFS-13948_ > Regex Link Type In Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount > Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HADOOP-15891.015.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HADOOP-15891.015.patch, HDFS-13948.001.patch, > HDFS-13948.002.patch, HDFS-13948.003.patch, HDFS-13948.004.patch, > HDFS-13948.005.patch, HDFS-13948.006.patch, HDFS-13948.007.patch, > HDFS-13948.008.patch, HDFS-13948.009.patch, HDFS-13948.011.patch, > HDFS-13948.012.patch, HDFS-13948.013.patch, HDFS-13948.014.patch, HDFS-13948_ > Regex Link Type In Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount > Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16705408#comment-16705408 ] zhenzhao wang commented on HADOOP-15891: [~lohit][~ctrezzo][~vrushalic] [~liuml07] [~manojg] [~arpiagariu] Could you please help to review it when you get a chance? Thanks. > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948.011.patch, HDFS-13948.012.patch, > HDFS-13948.013.patch, HDFS-13948.014.patch, HDFS-13948_ Regex Link Type In > Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.014.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948.011.patch, HDFS-13948.012.patch, > HDFS-13948.013.patch, HDFS-13948.014.patch, HDFS-13948_ Regex Link Type In > Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.013.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948.011.patch, HDFS-13948.012.patch, > HDFS-13948.013.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.012.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948.011.patch, HDFS-13948.012.patch, HDFS-13948_ > Regex Link Type In Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount > Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.011.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948.011.patch, HDFS-13948_ Regex Link Type In > Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: (was: HDFS-13948.009.patch) > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.009.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.009.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: (was: HDFS-13948.009.patch) > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: (was: HDFS-13948.009.patch) > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.009.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.009.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, > HDFS-13948.009.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.008.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948.008.patch, HDFS-13948_ > Regex Link Type In Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount > Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.007.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948_ Regex Link Type In > Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: (was: HDFS-13948.007.patch) > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.007.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948.007.patch, HDFS-13948_ Regex Link Type In > Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.006.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, > HDFS-13948.006.patch, HDFS-13948_ Regex Link Type In Mont Table-V0.pdf, > HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948.005.patch > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948.005.patch, HDFS-13948_ > Regex Link Type In Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount > Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669480#comment-16669480 ] zhenzhao wang commented on HADOOP-15891: Updated design doc on interceptors. [^HDFS-13948_ Regex Link Type In Mount Table-v1.pdf] > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948_ Regex Link Type In > Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15891) Provide Regex Based Mount Point In Inode Tree
[ https://issues.apache.org/jira/browse/HADOOP-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhenzhao wang updated HADOOP-15891: --- Attachment: HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > Provide Regex Based Mount Point In Inode Tree > - > > Key: HADOOP-15891 > URL: https://issues.apache.org/jira/browse/HADOOP-15891 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: zhenzhao wang >Assignee: zhenzhao wang >Priority: Major > Attachments: HDFS-13948.001.patch, HDFS-13948.002.patch, > HDFS-13948.003.patch, HDFS-13948.004.patch, HDFS-13948_ Regex Link Type In > Mont Table-V0.pdf, HDFS-13948_ Regex Link Type In Mount Table-v1.pdf > > > This jira is created to support regex based mount point in Inode Tree. We > noticed that mount point only support fixed target path. However, we might > have user cases when target needs to refer some fields from source. e.g. We > might want a mapping of /cluster1/user1 => /cluster1-dc1/user-nn-user1, we > want to refer `cluster` and `user` field in source to construct target. It's > impossible to archive this with current link type. Though we could set > one-to-one mapping, the mount table would become bloated if we have thousands > of users. Besides, a regex mapping would empower us more flexibility. So we > are going to build a regex based mount point which target could refer groups > from src regex mapping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
