[jira] [Commented] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[ https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138770#comment-16138770 ] Gary Gregory commented on VFS-640: -- Bernd, Are you available to implement this? Gary > Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad > --- > > Key: VFS-640 > URL: https://issues.apache.org/jira/browse/VFS-640 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.0, 2.1 >Reporter: L > > Currently (rev. 1802455) this method looks like this: > {code:java} > protected synchronized FileSystem getFileSystem(final FileName rootName, > final FileSystemOptions fileSystemOptions) > throws FileSystemException > { > FileSystem fs = findFileSystem(rootName, fileSystemOptions); > if (fs == null) > { > // Need to create the file system, and cache it > fs = doCreateFileSystem(rootName, fileSystemOptions); > addFileSystem(rootName, fs); > } > return fs; > } > {code} > Given that there is a single instance of a provider per scheme, a very slow > server that is being accessed first time will block *all* other threads > trying to access some other resources via the same provider. > We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We > do not mind the thread trying to access this method taking that long, but > because of that "synchronized" we end up having multiple threads accessing > unrelated servers for the same scheme blocked. > PS. Actually, it is not only AbstractOriginatingFileProvider. The same > "provider-scoped" lock is present in AbstractLayeredFileProvider, > TemporaryFileProvider, and UrlFileProvider -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[ https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138728#comment-16138728 ] Bernd Eckenfels commented on VFS-640: - One option would be to avoid slow server connections in the createFilesystem. A more general solution could be having a lock keyed by serverendpoint. > Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad > --- > > Key: VFS-640 > URL: https://issues.apache.org/jira/browse/VFS-640 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.0, 2.1 >Reporter: L > > Currently (rev. 1802455) this method looks like this: > {code:java} > protected synchronized FileSystem getFileSystem(final FileName rootName, > final FileSystemOptions fileSystemOptions) > throws FileSystemException > { > FileSystem fs = findFileSystem(rootName, fileSystemOptions); > if (fs == null) > { > // Need to create the file system, and cache it > fs = doCreateFileSystem(rootName, fileSystemOptions); > addFileSystem(rootName, fs); > } > return fs; > } > {code} > Given that there is a single instance of a provider per scheme, a very slow > server that is being accessed first time will block *all* other threads > trying to access some other resources via the same provider. > We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We > do not mind the thread trying to access this method taking that long, but > because of that "synchronized" we end up having multiple threads accessing > unrelated servers for the same scheme blocked. > PS. Actually, it is not only AbstractOriginatingFileProvider. The same > "provider-scoped" lock is present in AbstractLayeredFileProvider, > TemporaryFileProvider, and UrlFileProvider -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (CLI-275) Cannot get a full quoted argument through
[ https://issues.apache.org/jira/browse/CLI-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138528#comment-16138528 ] Csaba Skrabak edited comment on CLI-275 at 8/23/17 3:58 PM: [~glennm], your 4th example: {noformat} "'"~"'" {noformat} ...is fully interpreted by bash, that is why echo behaves the same way. Bash by the way interprets it the following way: 1. Double quotation mark: quoted string starts (first one.) 2. Single quotation mark: a literal apostrophe is passed over to the process since we are inside quotation. 3. Double quotation mark: first quoted string ends here. 4. Tilde. Has no special meaning, passing it over to the process. (It does have special meaning in Mac OS bash; but you seem to have tested it on Linux bash.) 5. Double quotation mark. Starting a quoted string again. This is a second quotation. 6. Single quotation mark. A literal apostrophe is passed over to the process since we are inside (the second) quotation. 7. Double quotation mark. End of the second quotation. 8. End. So as a result, we passed over an apostrophe in step 2, a tilde in step 4 and another apostrophe in step 6: {noformat} '~' {noformat} tada.wav was (Author: cskrabak): [~glennm], your 4th example: {noformat} "'"~"'" {noformat} ...is fully interpreted by bash, that is why echo behaves the same way. Bash by the way interprets it the following way: 1. Double quotation mark: quoted string starts (first one.) 2. Single quotation mark: a literal apostrophe is passed over to the process since we are inside quotation. 3. Double quotation mark: first quoted string ends here. 4. Tilde. Has no special meaning, passing it over to the process. 5. Double quotation mark. Starting a quoted string again. This is a second quotation. 6. Single quotation mark. A literal apostrophe is passed over to the process since we are inside (the second) quotation. 7. Double quotation mark. End of the second quotation. 8. End. So as a result, we passed over an apostrophe in step 2, a tilde in step 4 and another apostrophe in step 6: {noformat} '~' {noformat} tada.wav > Cannot get a full quoted argument through > - > > Key: CLI-275 > URL: https://issues.apache.org/jira/browse/CLI-275 > Project: Commons CLI > Issue Type: Bug >Reporter: Csaba Skrabak > > CLI-185 has been partially fixed but user still cannot pass in an argument > like "x" (the whole argument surrounded by one pair of double quotes.) If > user enters \"x\" then the quotes get eaten (btw. WHY THE HECK?!). But if > user enters \"\"x\"\" then both pairs of quotes are left intact. > The idea of removing the quotes is plain wrong, should be either forgotten > entirely or the argument should be parsed properly (rather than tinkering > from bug to bug with variations of find/substring call constructs) with the > possibility to escape and quote quotation marks. > Blocks: PHOENIX-3710 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (CLI-275) Cannot get a full quoted argument through
[ https://issues.apache.org/jira/browse/CLI-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138504#comment-16138504 ] Csaba Skrabak edited comment on CLI-275 at 8/23/17 3:56 PM: [~glennm], I saw the code that removes the double quotes. It's defined and deterministic. * It will remove only double quotes, does not react to single quotation marks in any way. * It removes only from the both ends of an argument. It does not remove them from the middle. If a quotation starts at the beginning of the argument but ends in the middle, or starts in the middle and ends at the end then it also stays in place. * It only processes what it gets. So any quotation mark and escape character that is processed by the shell will behave according to shell rules. Only those quotation marks are subject to Apache CLI processing that are escaped or quoted to the shell. * Note that the shell is not necessarily bash or Bourne Shell. It may as well be cmd.exe or things I don't know about. So this is a grammar that is defined and tested (although very annoying to bash users.) My main concern is that this grammar does not contain the string with a pair of double quotes on both ends. I.e., there is no way to end up in "xx" (including the quotation marks) no matter how hard the user escapes things. Passing in an argument like "xx" is exactly what Phoenix would need to interpret a table name case sensitively. What is even sadder in this situation is that only a backward incompatible change would solve the problem. was (Author: cskrabak): [~glennm], I saw the code that removes the double quotes. It's defined and deterministic. * It will remove only double quotes, does not react to single quotation marks in any way. * It removes only from the both ends of an argument. It does not remove them from the middle. If a quotation starts at the beginning of the argument but ends in the middle, or starts in the middle and ends at the end then it also stays in place. * It only processes what it gets. So any quotation mark and escape character that is processed by the shell will behave according to shell rules. Only those quotation marks are subject to Apache CLI processing that are escaped or quoted to the shell. * Note that the shell is not necessarily bash or Bourne Shell. It may as well be cmd.exe or things I don't know about. So this is a grammar that is defined and tested (although very annoying to bash users.) My main concern is that this grammar does not contain the string with a pair of double quotes on both ends. I.e., there is no way to end up in "xx" (including the quotation marks) no matter how hard the user escapes things. Passing in an argument like "xx" is exactly what Phoenix would need to interpret a table name case sensitively. > Cannot get a full quoted argument through > - > > Key: CLI-275 > URL: https://issues.apache.org/jira/browse/CLI-275 > Project: Commons CLI > Issue Type: Bug >Reporter: Csaba Skrabak > > CLI-185 has been partially fixed but user still cannot pass in an argument > like "x" (the whole argument surrounded by one pair of double quotes.) If > user enters \"x\" then the quotes get eaten (btw. WHY THE HECK?!). But if > user enters \"\"x\"\" then both pairs of quotes are left intact. > The idea of removing the quotes is plain wrong, should be either forgotten > entirely or the argument should be parsed properly (rather than tinkering > from bug to bug with variations of find/substring call constructs) with the > possibility to escape and quote quotation marks. > Blocks: PHOENIX-3710 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (CLI-275) Cannot get a full quoted argument through
[ https://issues.apache.org/jira/browse/CLI-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138528#comment-16138528 ] Csaba Skrabak edited comment on CLI-275 at 8/23/17 3:48 PM: [~glennm], your 4th example: {noformat} "'"~"'" {noformat} ...is fully interpreted by bash, that is why echo behaves the same way. Bash by the way interprets it the following way: 1. Double quotation mark: quoted string starts (first one.) 2. Single quotation mark: a literal apostrophe is passed over to the process since we are inside quotation. 3. Double quotation mark: first quoted string ends here. 4. Tilde. Has no special meaning, passing it over to the process. 5. Double quotation mark. Starting a quoted string again. This is a second quotation. 6. Single quotation mark. A literal apostrophe is passed over to the process since we are inside (the second) quotation. 7. Double quotation mark. End of the second quotation. 8. End. So as a result, we passed over an apostrophe in step 2, a tilde in step 4 and another apostrophe in step 6: {noformat} '~' {noformat} tada.wav was (Author: cskrabak): [~glennm], your 4th example: {noformat} "'"~"'" {noformat} ...is fully interpreted by bash, thet is why echo behaves the same way. Bash by the way interprets it the following way: 1. Double quotation mark: quoted string starts (first one.) 2. Single quotation mark: a literal apostrophe is passed over to the process since we are inside quotation. 3. Double quotation mark: first quoted string ends here. 4. Tilde. Has no special meaning, passing it over to the process. 5. Double quotation mark. Starting a quoted string again. This is a second quotation. 6. Single quotation mark. A literal apostrophe is passed over to the process since we are inside (the second) quotation. 7. Double quotation mark. End of the second quotation. 8. End. So as a result, we passed over an apostrophe in step 2, a tilde in step 4 and another apostrophe in step 6: {noformat} '~' {noformat} tada.wav > Cannot get a full quoted argument through > - > > Key: CLI-275 > URL: https://issues.apache.org/jira/browse/CLI-275 > Project: Commons CLI > Issue Type: Bug >Reporter: Csaba Skrabak > > CLI-185 has been partially fixed but user still cannot pass in an argument > like "x" (the whole argument surrounded by one pair of double quotes.) If > user enters \"x\" then the quotes get eaten (btw. WHY THE HECK?!). But if > user enters \"\"x\"\" then both pairs of quotes are left intact. > The idea of removing the quotes is plain wrong, should be either forgotten > entirely or the argument should be parsed properly (rather than tinkering > from bug to bug with variations of find/substring call constructs) with the > possibility to escape and quote quotation marks. > Blocks: PHOENIX-3710 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CLI-275) Cannot get a full quoted argument through
[ https://issues.apache.org/jira/browse/CLI-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138528#comment-16138528 ] Csaba Skrabak commented on CLI-275: --- [~glennm], your 4th example: {noformat} "'"~"'" {noformat} ...is fully interpreted by bash, thet is why echo behaves the same way. Bash by the way interprets it the following way: 1. Double quotation mark: quoted string starts (first one.) 2. Single quotation mark: a literal apostrophe is passed over to the process since we are inside quotation. 3. Double quotation mark: first quoted string ends here. 4. Tilde. Has no special meaning, passing it over to the process. 5. Double quotation mark. Starting a quoted string again. This is a second quotation. 6. Single quotation mark. A literal apostrophe is passed over to the process since we are inside (the second) quotation. 7. Double quotation mark. End of the second quotation. 8. End. So as a result, we passed over an apostrophe in step 2, a tilde in step 4 and another apostrophe in step 6: {noformat} '~' {noformat} tada.wav > Cannot get a full quoted argument through > - > > Key: CLI-275 > URL: https://issues.apache.org/jira/browse/CLI-275 > Project: Commons CLI > Issue Type: Bug >Reporter: Csaba Skrabak > > CLI-185 has been partially fixed but user still cannot pass in an argument > like "x" (the whole argument surrounded by one pair of double quotes.) If > user enters \"x\" then the quotes get eaten (btw. WHY THE HECK?!). But if > user enters \"\"x\"\" then both pairs of quotes are left intact. > The idea of removing the quotes is plain wrong, should be either forgotten > entirely or the argument should be parsed properly (rather than tinkering > from bug to bug with variations of find/substring call constructs) with the > possibility to escape and quote quotation marks. > Blocks: PHOENIX-3710 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CLI-275) Cannot get a full quoted argument through
[ https://issues.apache.org/jira/browse/CLI-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138504#comment-16138504 ] Csaba Skrabak commented on CLI-275: --- [~glennm], I saw the code that removes the double quotes. It's defined and deterministic. * It will remove only double quotes, does not react to single quotation marks in any way. * It removes only from the both ends of an argument. It does not remove them from the middle. If a quotation starts at the beginning of the argument but ends in the middle, or starts in the middle and ends at the end then it also stays in place. * It only processes what it gets. So any quotation mark and escape character that is processed by the shell will behave according to shell rules. Only those quotation marks are subject to Apache CLI processing that are escaped or quoted to the shell. * Note that the shell is not necessarily bash or Bourne Shell. It may as well be cmd.exe or things I don't know about. So this is a grammar that is defined and tested (although very annoying to bash users.) My main concern is that this grammar does not contain the string with a pair of double quotes on both ends. I.e., there is no way to end up in "xx" (including the quotation marks) no matter how hard the user escapes things. Passing in an argument like "xx" is exactly what Phoenix would need to interpret a table name case sensitively. > Cannot get a full quoted argument through > - > > Key: CLI-275 > URL: https://issues.apache.org/jira/browse/CLI-275 > Project: Commons CLI > Issue Type: Bug >Reporter: Csaba Skrabak > > CLI-185 has been partially fixed but user still cannot pass in an argument > like "x" (the whole argument surrounded by one pair of double quotes.) If > user enters \"x\" then the quotes get eaten (btw. WHY THE HECK?!). But if > user enters \"\"x\"\" then both pairs of quotes are left intact. > The idea of removing the quotes is plain wrong, should be either forgotten > entirely or the argument should be parsed properly (rather than tinkering > from bug to bug with variations of find/substring call constructs) with the > possibility to escape and quote quotation marks. > Blocks: PHOENIX-3710 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[ https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138447#comment-16138447 ] Gary Gregory commented on VFS-640: -- What solution do you propose that would still provide thread safety? > Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad > --- > > Key: VFS-640 > URL: https://issues.apache.org/jira/browse/VFS-640 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.0, 2.1 >Reporter: L > > Currently (rev. 1802455) this method looks like this: > {code:java} > protected synchronized FileSystem getFileSystem(final FileName rootName, > final FileSystemOptions fileSystemOptions) > throws FileSystemException > { > FileSystem fs = findFileSystem(rootName, fileSystemOptions); > if (fs == null) > { > // Need to create the file system, and cache it > fs = doCreateFileSystem(rootName, fileSystemOptions); > addFileSystem(rootName, fs); > } > return fs; > } > {code} > Given that there is a single instance of a provider per scheme, a very slow > server that is being accessed first time will block *all* other threads > trying to access some other resources via the same provider. > We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We > do not mind the thread trying to access this method taking that long, but > because of that "synchronized" we end up having multiple threads accessing > unrelated servers for the same scheme blocked. > PS. Actually, it is not only AbstractOriginatingFileProvider. The same > "provider-scoped" lock is present in AbstractLayeredFileProvider, > TemporaryFileProvider, and UrlFileProvider -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[ https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L updated VFS-640: -- Description: Currently (rev. 1802455) this method looks like this: {code:java} protected synchronized FileSystem getFileSystem(final FileName rootName, final FileSystemOptions fileSystemOptions) throws FileSystemException { FileSystem fs = findFileSystem(rootName, fileSystemOptions); if (fs == null) { // Need to create the file system, and cache it fs = doCreateFileSystem(rootName, fileSystemOptions); addFileSystem(rootName, fs); } return fs; } {code} Given that there is a single instance of a provider per scheme, a very slow server that is being accessed first time will block *all* other threads trying to access some other resources via the same provider. We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We do not mind the thread trying to access this method taking that long, but because of that "synchronized" we end up having multiple threads accessing unrelated servers for the same scheme blocked. PS. Actually, it is not only AbstractOriginatingFileProvider. The same "provider-scoped" lock is present in AbstractLayeredFileProvider, TemporaryFileProvider, and UrlFileProvider was: Currently (rev. 1802455) this method looks like this: {code:java} protected synchronized FileSystem getFileSystem(final FileName rootName, final FileSystemOptions fileSystemOptions) throws FileSystemException { FileSystem fs = findFileSystem(rootName, fileSystemOptions); if (fs == null) { // Need to create the file system, and cache it fs = doCreateFileSystem(rootName, fileSystemOptions); addFileSystem(rootName, fs); } return fs; } {code} Given that there is a single instance of a provider per scheme, a very slow server that is being accessed first time will block *all* other threads trying to access some other resources via the same provider. We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We do not mind the thread trying to access this method taking that long, but because of that "synchronized" we end up having multiple threads accessing unrelated servers for the same scheme blocked. Actually, it is not only > Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad > --- > > Key: VFS-640 > URL: https://issues.apache.org/jira/browse/VFS-640 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.0, 2.1 >Reporter: L > > Currently (rev. 1802455) this method looks like this: > {code:java} > protected synchronized FileSystem getFileSystem(final FileName rootName, > final FileSystemOptions fileSystemOptions) > throws FileSystemException > { > FileSystem fs = findFileSystem(rootName, fileSystemOptions); > if (fs == null) > { > // Need to create the file system, and cache it > fs = doCreateFileSystem(rootName, fileSystemOptions); > addFileSystem(rootName, fs); > } > return fs; > } > {code} > Given that there is a single instance of a provider per scheme, a very slow > server that is being accessed first time will block *all* other threads > trying to access some other resources via the same provider. > We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We > do not mind the thread trying to access this method taking that long, but > because of that "synchronized" we end up having multiple threads accessing > unrelated servers for the same scheme blocked. > PS. Actually, it is not only AbstractOriginatingFileProvider. The same > "provider-scoped" lock is present in AbstractLayeredFileProvider, > TemporaryFileProvider, and UrlFileProvider -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[ https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L updated VFS-640: -- Description: Currently (rev. 1802455) this method looks like this: {code:java} protected synchronized FileSystem getFileSystem(final FileName rootName, final FileSystemOptions fileSystemOptions) throws FileSystemException { FileSystem fs = findFileSystem(rootName, fileSystemOptions); if (fs == null) { // Need to create the file system, and cache it fs = doCreateFileSystem(rootName, fileSystemOptions); addFileSystem(rootName, fs); } return fs; } {code} Given that there is a single instance of a provider per scheme, a very slow server that is being accessed first time will block *all* other threads trying to access some other resources via the same provider. We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We do not mind the thread trying to access this method taking that long, but because of that "synchronized" we end up having multiple threads accessing unrelated servers for the same scheme blocked. Actually, it is not only was: Currently (rev. 1802455) this method looks like this: {code:java} protected synchronized FileSystem getFileSystem(final FileName rootName, final FileSystemOptions fileSystemOptions) throws FileSystemException { FileSystem fs = findFileSystem(rootName, fileSystemOptions); if (fs == null) { // Need to create the file system, and cache it fs = doCreateFileSystem(rootName, fileSystemOptions); addFileSystem(rootName, fs); } return fs; } {code} Given that there is a single instance of a provider per scheme, a very slow server that is being accessed first time will block *all* other threads trying to access some other resources via the same provider. We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We do not mind the thread trying to access this method taking that long, but because of that "synchronized" we end up having multiple threads accessing unrelated servers for the same scheme blocked. > Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad > --- > > Key: VFS-640 > URL: https://issues.apache.org/jira/browse/VFS-640 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.0, 2.1 >Reporter: L > > Currently (rev. 1802455) this method looks like this: > {code:java} > protected synchronized FileSystem getFileSystem(final FileName rootName, > final FileSystemOptions fileSystemOptions) > throws FileSystemException > { > FileSystem fs = findFileSystem(rootName, fileSystemOptions); > if (fs == null) > { > // Need to create the file system, and cache it > fs = doCreateFileSystem(rootName, fileSystemOptions); > addFileSystem(rootName, fs); > } > return fs; > } > {code} > Given that there is a single instance of a provider per scheme, a very slow > server that is being accessed first time will block *all* other threads > trying to access some other resources via the same provider. > We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We > do not mind the thread trying to access this method taking that long, but > because of that "synchronized" we end up having multiple threads accessing > unrelated servers for the same scheme blocked. > Actually, it is not only -- This message was sent by Atlassian JIRA (v6.4.14#64029)