Re: [RFC] Server Dictated Configuration
On Wed, Feb 1, 2012 at 7:38 AM, Julian Foad julianf...@btopenworld.com wrote: Hi Paul. Thanks for indulging my enquiries. Paul Burba wrote: Julian Foad wrote: Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this: /subversion svn:i:ignore = *.o *.obj *.a *.lib ... /subversion/trunk/tests svn:i:ignore = *.obj *.a *.lib ... ... which can be done hierarchically; but every such override at a subtree level duplicates much of the information that was provided at the '/subversion' level, which means that whenever we modify the base setting we probably want to look through the whole repository and modify all the subtree settings in the same way. Note that my use case in this example is about wanting to *remove* one pattern from the default list, in a specific subdirectory. On the flip side, if the value of svn:i:ignore on /subversion/trunk/tests didn't override, but rather appended to, the inherited value from /subversion, ... then there would be no way to achieve my use case by setting svn:i:ignore on the 'tests' subdirectory, unless you provide some as-yet-unmentioned alternative ... then if we change the base value then we *still* need to look through the whole repository and modify all the subtree settings so that we are no subtree is appending a value we *don't* want. ... so I don't follow what you're saying here. [...] Anyhow, while we might currently have different ideas on how best to implement ignores via inheritable props, your point about taking us into the future is a valid one. I'm still not certain ignores needs to use both explicit and inherited values, but certainly some future inheritable property might need both. To that end I tweaked the suggested APIs in the wiki to provide this functionality -- the callers can decide what they need. But how would that work, concretely? I'm asking because the sort of scheme I had in mind was one where the client would define some syntax to be used inside an svn:i:ignore property value to specify which patterns should be added or removed. I don't see how this revised API could support that scheme recursively. Maybe you have a different scheme in mind as a use case. Hi Julian, Does the latest version of the proposed APIs address your concerns? We can ask for a path's explicit mergeinfo and the inheritable properties it inherits from its path-wise ancestors (not just the path's closest parent, but *all* its parents going all the way to the root of the repository). It seems to me this is sufficient to implement any of the inheritable property rules we might dream up for a particular inheritable property. Paul It's clearly quite tricky to design a useful inherited properties system. The end result need not be particularly complex, but it is hard to tell by inspection whether a given design proposal would end up meeting real-life needs in a reasonably worthwhile way. If we are going to explore the inherited properties idea further, then we need to explore how we'd actually use the inherited properties in a non-trivial example. That's why my original question was (rephrased): What would the total design look like, for achieving some particular example of non-trivial end-user behaviour that could be facilitated by using inherited properties? Last time I asked the question, I meant to suggest exploring an inheritable reimplementation of 'ignore patterns' as an example, but the discussion got immediately sidetracked onto how we might implement an ignore-patterns system that extends and is backward compatible with the existing non-inheritable 'svn:ignore' property. That's important, of course, but not a particularly good way to explore the inherited properties design itself. Perhaps it would be less confusing to base an example on some new, made-up feature. As I noted before, it is not a requirement to be able to use inheritable properties with multi-element values (such as the current multi-line syntax of svn:ignore) and expect to be able to override/add/subtract single elements. We could require, say, that any semantic appending/overriding/subtracting that may be required should be mapped to whole-property operations. Then, for example, an inheritable reimplementation of ignore-patterns could make use of multiple props such as {svn:i:ignore:*.o = yes}, one prop per pattern. A subdirectory could subtract a particular pattern by setting {svn:i:ignore:*.o = no}. And to make clear that this last thought is not yet a complete solution, at this point I'd ask how a subdirectory could specify that all ignore patterns should be cleared/disabled without having to know what all the patterns are. - Julian
Re: [RFC] Server Dictated Configuration
Hi Paul. Thanks for indulging my enquiries. Paul Burba wrote: Julian Foad wrote: Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this: /subversion svn:i:ignore = *.o *.obj *.a *.lib ... /subversion/trunk/tests svn:i:ignore = *.obj *.a *.lib ... ... which can be done hierarchically; but every such override at a subtree level duplicates much of the information that was provided at the '/subversion' level, which means that whenever we modify the base setting we probably want to look through the whole repository and modify all the subtree settings in the same way. Note that my use case in this example is about wanting to *remove* one pattern from the default list, in a specific subdirectory. On the flip side, if the value of svn:i:ignore on /subversion/trunk/tests didn't override, but rather appended to, the inherited value from /subversion, ... then there would be no way to achieve my use case by setting svn:i:ignore on the 'tests' subdirectory, unless you provide some as-yet-unmentioned alternative ... then if we change the base value then we *still* need to look through the whole repository and modify all the subtree settings so that we are no subtree is appending a value we *don't* want. ... so I don't follow what you're saying here. [...] Anyhow, while we might currently have different ideas on how best to implement ignores via inheritable props, your point about taking us into the future is a valid one. I'm still not certain ignores needs to use both explicit and inherited values, but certainly some future inheritable property might need both. To that end I tweaked the suggested APIs in the wiki to provide this functionality -- the callers can decide what they need. But how would that work, concretely? I'm asking because the sort of scheme I had in mind was one where the client would define some syntax to be used inside an svn:i:ignore property value to specify which patterns should be added or removed. I don't see how this revised API could support that scheme recursively. Maybe you have a different scheme in mind as a use case. It's clearly quite tricky to design a useful inherited properties system. The end result need not be particularly complex, but it is hard to tell by inspection whether a given design proposal would end up meeting real-life needs in a reasonably worthwhile way. If we are going to explore the inherited properties idea further, then we need to explore how we'd actually use the inherited properties in a non-trivial example. That's why my original question was (rephrased): What would the total design look like, for achieving some particular example of non-trivial end-user behaviour that could be facilitated by using inherited properties? Last time I asked the question, I meant to suggest exploring an inheritable reimplementation of 'ignore patterns' as an example, but the discussion got immediately sidetracked onto how we might implement an ignore-patterns system that extends and is backward compatible with the existing non-inheritable 'svn:ignore' property. That's important, of course, but not a particularly good way to explore the inherited properties design itself. Perhaps it would be less confusing to base an example on some new, made-up feature. As I noted before, it is not a requirement to be able to use inheritable properties with multi-element values (such as the current multi-line syntax of svn:ignore) and expect to be able to override/add/subtract single elements. We could require, say, that any semantic appending/overriding/subtracting that may be required should be mapped to whole-property operations. Then, for example, an inheritable reimplementation of ignore-patterns could make use of multiple props such as {svn:i:ignore:*.o = yes}, one prop per pattern. A subdirectory could subtract a particular pattern by setting {svn:i:ignore:*.o = no}. And to make clear that this last thought is not yet a complete solution, at this point I'd ask how a subdirectory could specify that all ignore patterns should be cleared/disabled without having to know what all the patterns are. - Julian
Re: [RFC] Server Dictated Configuration
On Wed, Feb 1, 2012 at 1:38 PM, Julian Foad julianf...@btopenworld.com wrote: Hi Paul. Thanks for indulging my enquiries. Paul Burba wrote: Julian Foad wrote: Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this: /subversion svn:i:ignore = *.o *.obj *.a *.lib ... /subversion/trunk/tests svn:i:ignore = *.obj *.a *.lib ... ... which can be done hierarchically; but every such override at a subtree level duplicates much of the information that was provided at the '/subversion' level, which means that whenever we modify the base setting we probably want to look through the whole repository and modify all the subtree settings in the same way. Note that my use case in this example is about wanting to *remove* one pattern from the default list, in a specific subdirectory. On the flip side, if the value of svn:i:ignore on /subversion/trunk/tests didn't override, but rather appended to, the inherited value from /subversion, ... then there would be no way to achieve my use case by setting svn:i:ignore on the 'tests' subdirectory, unless you provide some as-yet-unmentioned alternative ... then if we change the base value then we *still* need to look through the whole repository and modify all the subtree settings so that we are no subtree is appending a value we *don't* want. ... so I don't follow what you're saying here. [...] Anyhow, while we might currently have different ideas on how best to implement ignores via inheritable props, your point about taking us into the future is a valid one. I'm still not certain ignores needs to use both explicit and inherited values, but certainly some future inheritable property might need both. To that end I tweaked the suggested APIs in the wiki to provide this functionality -- the callers can decide what they need. But how would that work, concretely? I'm asking because the sort of scheme I had in mind was one where the client would define some syntax to be used inside an svn:i:ignore property value to specify which patterns should be added or removed. I don't see how this revised API could support that scheme recursively. Maybe you have a different scheme in mind as a use case. It's clearly quite tricky to design a useful inherited properties system. The end result need not be particularly complex, but it is hard to tell by inspection whether a given design proposal would end up meeting real-life needs in a reasonably worthwhile way. If we are going to explore the inherited properties idea further, then we need to explore how we'd actually use the inherited properties in a non-trivial example. That's why my original question was (rephrased): What would the total design look like, for achieving some particular example of non-trivial end-user behaviour that could be facilitated by using inherited properties? Last time I asked the question, I meant to suggest exploring an inheritable reimplementation of 'ignore patterns' as an example, but the discussion got immediately sidetracked onto how we might implement an ignore-patterns system that extends and is backward compatible with the existing non-inheritable 'svn:ignore' property. That's important, of course, but not a particularly good way to explore the inherited properties design itself. Perhaps it would be less confusing to base an example on some new, made-up feature. As I noted before, it is not a requirement to be able to use inheritable properties with multi-element values (such as the current multi-line syntax of svn:ignore) and expect to be able to override/add/subtract single elements. We could require, say, that any semantic appending/overriding/subtracting that may be required should be mapped to whole-property operations. Then, for example, an inheritable reimplementation of ignore-patterns could make use of multiple props such as {svn:i:ignore:*.o = yes}, one prop per pattern. A subdirectory could subtract a particular pattern by setting {svn:i:ignore:*.o = no}. And to make clear that this last thought is not yet a complete solution, at this point I'd ask how a subdirectory could specify that all ignore patterns should be cleared/disabled without having to know what all the patterns are. Perhaps this issue can be deferred if the API would allow you to get / cache all the inheritable props, up to the repository root (since they are always readable anyway, regardless of authz). Not only up to the nearest path-wise ancestor that has the property, but give me everything up to the root. Then the concrete consumer of the prop can always add special syntax and corresponding inheritance / override / append behavior specific to its uses (a special syntax that defines how the multiple inherited props are to be overlayed). OTOH, a standardized syntax and corresponding defined behavior might be nice though. I'm just a
Re: [RFC] Server Dictated Configuration
On Wed, Feb 1, 2012 at 7:38 AM, Julian Foad julianf...@btopenworld.com wrote: Hi Paul. Thanks for indulging my enquiries. Paul Burba wrote: Julian Foad wrote: Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this: /subversion svn:i:ignore = *.o *.obj *.a *.lib ... /subversion/trunk/tests svn:i:ignore = *.obj *.a *.lib ... ... which can be done hierarchically; but every such override at a subtree level duplicates much of the information that was provided at the '/subversion' level, which means that whenever we modify the base setting we probably want to look through the whole repository and modify all the subtree settings in the same way. Note that my use case in this example is about wanting to *remove* one pattern from the default list, in a specific subdirectory. On the flip side, if the value of svn:i:ignore on /subversion/trunk/tests didn't override, but rather appended to, the inherited value from /subversion, ... then there would be no way to achieve my use case by setting svn:i:ignore on the 'tests' subdirectory, unless you provide some as-yet-unmentioned alternative ... then if we change the base value then we *still* need to look through the whole repository and modify all the subtree settings so that we are no subtree is appending a value we *don't* want. ... so I don't follow what you're saying here. [...] Anyhow, while we might currently have different ideas on how best to implement ignores via inheritable props, your point about taking us into the future is a valid one. I'm still not certain ignores needs to use both explicit and inherited values, but certainly some future inheritable property might need both. To that end I tweaked the suggested APIs in the wiki to provide this functionality -- the callers can decide what they need. But how would that work, concretely? I'm asking because the sort of scheme I had in mind was one where the client would define some syntax to be used inside an svn:i:ignore property value to specify which patterns should be added or removed. I don't see how this revised API could support that scheme recursively. Ah yes, my suggested API changes are bogus since they only handle the case of a single child and parent both of which have a given inheritable property. But what if there are grand-parents with the same inheritable property? Obviously we need to take those into account if we plan to support the concept of a path having an explicit inheritable property *and* inheriting the same property from a parent. I updated the wiki to reflect this. Paul Maybe you have a different scheme in mind as a use case. It's clearly quite tricky to design a useful inherited properties system. The end result need not be particularly complex, but it is hard to tell by inspection whether a given design proposal would end up meeting real-life needs in a reasonably worthwhile way. If we are going to explore the inherited properties idea further, then we need to explore how we'd actually use the inherited properties in a non-trivial example. That's why my original question was (rephrased): What would the total design look like, for achieving some particular example of non-trivial end-user behaviour that could be facilitated by using inherited properties? Last time I asked the question, I meant to suggest exploring an inheritable reimplementation of 'ignore patterns' as an example, but the discussion got immediately sidetracked onto how we might implement an ignore-patterns system that extends and is backward compatible with the existing non-inheritable 'svn:ignore' property. That's important, of course, but not a particularly good way to explore the inherited properties design itself. Perhaps it would be less confusing to base an example on some new, made-up feature. As I noted before, it is not a requirement to be able to use inheritable properties with multi-element values (such as the current multi-line syntax of svn:ignore) and expect to be able to override/add/subtract single elements. We could require, say, that any semantic appending/overriding/subtracting that may be required should be mapped to whole-property operations. Then, for example, an inheritable reimplementation of ignore-patterns could make use of multiple props such as {svn:i:ignore:*.o = yes}, one prop per pattern. A subdirectory could subtract a particular pattern by setting {svn:i:ignore:*.o = no}. And to make clear that this last thought is not yet a complete solution, at this point I'd ask how a subdirectory could specify that all ignore patterns should be cleared/disabled without having to know what all the patterns are. - Julian
Re: [RFC] Server Dictated Configuration
On Wed, Feb 1, 2012 at 8:46 AM, Johan Corveleyn jcor...@gmail.com wrote: On Wed, Feb 1, 2012 at 1:38 PM, Julian Foad julianf...@btopenworld.com wrote: Hi Paul. Thanks for indulging my enquiries. Paul Burba wrote: Julian Foad wrote: Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this: /subversion svn:i:ignore = *.o *.obj *.a *.lib ... /subversion/trunk/tests svn:i:ignore = *.obj *.a *.lib ... ... which can be done hierarchically; but every such override at a subtree level duplicates much of the information that was provided at the '/subversion' level, which means that whenever we modify the base setting we probably want to look through the whole repository and modify all the subtree settings in the same way. Note that my use case in this example is about wanting to *remove* one pattern from the default list, in a specific subdirectory. On the flip side, if the value of svn:i:ignore on /subversion/trunk/tests didn't override, but rather appended to, the inherited value from /subversion, ... then there would be no way to achieve my use case by setting svn:i:ignore on the 'tests' subdirectory, unless you provide some as-yet-unmentioned alternative ... then if we change the base value then we *still* need to look through the whole repository and modify all the subtree settings so that we are no subtree is appending a value we *don't* want. ... so I don't follow what you're saying here. [...] Anyhow, while we might currently have different ideas on how best to implement ignores via inheritable props, your point about taking us into the future is a valid one. I'm still not certain ignores needs to use both explicit and inherited values, but certainly some future inheritable property might need both. To that end I tweaked the suggested APIs in the wiki to provide this functionality -- the callers can decide what they need. But how would that work, concretely? I'm asking because the sort of scheme I had in mind was one where the client would define some syntax to be used inside an svn:i:ignore property value to specify which patterns should be added or removed. I don't see how this revised API could support that scheme recursively. Maybe you have a different scheme in mind as a use case. It's clearly quite tricky to design a useful inherited properties system. The end result need not be particularly complex, but it is hard to tell by inspection whether a given design proposal would end up meeting real-life needs in a reasonably worthwhile way. If we are going to explore the inherited properties idea further, then we need to explore how we'd actually use the inherited properties in a non-trivial example. That's why my original question was (rephrased): What would the total design look like, for achieving some particular example of non-trivial end-user behaviour that could be facilitated by using inherited properties? Last time I asked the question, I meant to suggest exploring an inheritable reimplementation of 'ignore patterns' as an example, but the discussion got immediately sidetracked onto how we might implement an ignore-patterns system that extends and is backward compatible with the existing non-inheritable 'svn:ignore' property. That's important, of course, but not a particularly good way to explore the inherited properties design itself. Perhaps it would be less confusing to base an example on some new, made-up feature. As I noted before, it is not a requirement to be able to use inheritable properties with multi-element values (such as the current multi-line syntax of svn:ignore) and expect to be able to override/add/subtract single elements. We could require, say, that any semantic appending/overriding/subtracting that may be required should be mapped to whole-property operations. Then, for example, an inheritable reimplementation of ignore-patterns could make use of multiple props such as {svn:i:ignore:*.o = yes}, one prop per pattern. A subdirectory could subtract a particular pattern by setting {svn:i:ignore:*.o = no}. And to make clear that this last thought is not yet a complete solution, at this point I'd ask how a subdirectory could specify that all ignore patterns should be cleared/disabled without having to know what all the patterns are. Perhaps this issue can be deferred if the API would allow you to get / cache all the inheritable props, up to the repository root (since they are always readable anyway, regardless of authz). Not only up to the nearest path-wise ancestor that has the property, but give me everything up to the root. Yes, this is what I should have done when tweaking the APIs last night, I had my head buried in the overly simple case of child-parent and forgot about those other pesky ancestors. I adjusted the suggested APIs in the wiki so they can get *all*
Re: [RFC] Server Dictated Configuration
Paul Burba wrote: Julian Foad wrote: The ability to see the inherited value and then merge in a child-defined value (adding/subtracting/overriding semantic sub-elements within the property value) is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. Why do we need to subtract and override? [...] 5) [...] we take a path's inherited (or explicit) svn:i:ignores property value, the svn:ignore property (if any) on a path's parent directory, and the global-ignores runtime config value and append all three together to get the final answer on what to ignore. I take it you view this as insufficient? This hybrid approach to defining the ignore patterns by means of one inheritable property and a different non-inheritable property gives us in total the ability to both override and append. Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this: /subversion svn:i:ignore = *.o *.obj *.a *.lib ... /subversion/trunk/tests svn:i:ignore = *.obj *.a *.lib ... ... which can be done hierarchically; but every such override at a subtree level duplicates much of the information that was provided at the '/subversion' level, which means that whenever we modify the base setting we probably want to look through the whole repository and modify all the subtree settings in the same way. Appending is done by a non-inheritable svn:ignore property, like this: /subversion/trunk/tests/libsvn_client svn:ignore = *-test *-test.exe *-tester *-tester.exe /subversion/trunk/tests/libsvn_delta svn:ignore = *-test *-test.exe *-tester *-tester.exe /subversion/trunk/tests/libsvn_diff svn:ignore = *-test *-test.exe *-tester *-tester.exe /subversion/trunk/tests/libsvn_... ... which can't be done hierarchically: we can't append the '*-test...' patterns just once at the '/subversion/trunk/tests/' directory level. So the result is better than what we have now, for ignore patterns. If that's all we need, fair enough. But if we're designing a feature to take us into the future, not just for svn:ignore, and if that's the best general usage pattern we can achieve, I feel we ought to be aiming for better. - Julian
Re: [RFC] Server Dictated Configuration
On Tue, Jan 31, 2012 at 12:48 PM, Julian Foad julianf...@btopenworld.com wrote: Paul Burba wrote: Julian Foad wrote: The ability to see the inherited value and then merge in a child-defined value (adding/subtracting/overriding semantic sub-elements within the property value) is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. Why do we need to subtract and override? [...] 5) [...] we take a path's inherited (or explicit) svn:i:ignores property value, the svn:ignore property (if any) on a path's parent directory, and the global-ignores runtime config value and append all three together to get the final answer on what to ignore. I take it you view this as insufficient? This hybrid approach to defining the ignore patterns by means of one inheritable property and a different non-inheritable property gives us in total the ability to both override and append. Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this: /subversion svn:i:ignore = *.o *.obj *.a *.lib ... /subversion/trunk/tests svn:i:ignore = *.obj *.a *.lib ... ... which can be done hierarchically; but every such override at a subtree level duplicates much of the information that was provided at the '/subversion' level, which means that whenever we modify the base setting we probably want to look through the whole repository and modify all the subtree settings in the same way. On the flip side, if the value of svn:i:ignore on /subversion/trunk/tests didn't override, but rather appended to, the inherited value from /subversion, then if we change the base value then we *still* need to look through the whole repository and modify all the subtree settings so that we are no subtree is appending a value we *don't* want. Append or override, there's not getting around the administrative hassle of subtree properties. Appending is done by a non-inheritable svn:ignore property, like this: /subversion/trunk/tests/libsvn_client svn:ignore = *-test *-test.exe *-tester *-tester.exe /subversion/trunk/tests/libsvn_delta svn:ignore = *-test *-test.exe *-tester *-tester.exe /subversion/trunk/tests/libsvn_diff svn:ignore = *-test *-test.exe *-tester *-tester.exe /subversion/trunk/tests/libsvn_... ... which can't be done hierarchically: we can't append the '*-test...' patterns just once at the '/subversion/trunk/tests/' directory level. So the result is better than what we have now, for ignore patterns. If that's all we need, fair enough. But if we're designing a feature to take us into the future, not just for svn:ignore, and if that's the best general usage pattern we can achieve, I feel we ought to be aiming for better. Anyhow, while we might currently have different ideas on how best to implement ignores via inheritable props, your point about taking us into the future is a valid one. I'm still not certain ignores needs to use both explicit and inherited values, but certainly some future inheritable property might need both. To that end I tweaked the suggested APIs in the wiki to provide this functionality -- the callers can decide what they need. Paul - Julian
Re: [RFC] Server Dictated Configuration
On 01/27/2012 04:38 PM, Paul Burba wrote: Now let's say we implement inheritable properties as I described in the wiki and want to use an inheritable property to supplement the existing mechanisms for svn:ignores/global-ignores. Isn't that as simple as this? 4) We add a new reserved inheritable property svn:i:ignores which has the exact same format as svn:ignore. 5) Again assuming a WC operation, we take a path's inherited (or explicit) svn:i:ignores property value, the svn:ignore property (if any) on a path's parent directory, and the global-ignores runtime config value and append all three together to get the final answer on what to ignore. I take it you view this as insufficient? The question wasn't really aimed my way, but this seems perfectly sufficient to me. -- C. Michael Pilato cmpil...@collab.net CollabNet www.collab.net Distributed Development On Demand signature.asc Description: OpenPGP digital signature
Re: [RFC] Server Dictated Configuration
Branko Čibej wrote: On 27.01.2012 12:53, Julian Foad wrote: Branko Čibej wrote: On 27.01.2012 11:50, Julian Foad wrote: We need to see how we'd implement a reasonable system of svn:ignores and auto-props using the proposed inheritable properties system. The ability to see the inherited value and then merge in a child-defined value [...] is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. [...] No, you need to give the inherited properties that carry server-dictated configuration a different name, don't you think? Whether the merging is then done server-side or client-side is almost a bikeshed. I'm not quite sure what you mean. Could you give a specific example? [...] One way to achieve server-dictated configuration of ignores would be to make the server control the 'global-ignores' [config setting]. But for the purposes of exploring inheritable properties, let's ignore the 'global-ignores' config setting and assume that we're going to control the ignores through (inherited) properties alone. [...] Heh, but I fail to see a semantic difference between the two cases. :) An inherited properties design implies client-side setting of the inherited properties, whereas the design for server-dictated configuration implies that setting will be done server-side by an administrator. For either approach, I would ask: how would you go about setting up a useful hierarchy of ignore patterns? In the server-side case, you can say we'll just start with a simple config file format and defer that problem; somebody can design a more powerful config system for the administrator to use, later. So I asked specifically about how one would conveniently define ignore-patterns hierarchically in a generally useful inherited properties design. Since the server-dictated global-ignores would only apply to a certain subtree in the repository, it would /already/ behave as if it were an inherited svn:ignore property, and what's more, would be implicitly merged by existing client implementation with any svn:ignore properties that subtree happens to contain. No. The way I read the proposed 'server-dictated config' scheme, it didn't include a way to configure different values for 'global-ignores' to apply to different directories inside the WC, only for transmitting a single value of 'global-ignores' which could depend on the root directory of the WC. But anyway, my point was to explore how useful the inherited properties idea would be in general, using ignore patterns as an example. If you're suggesting that this example of an inherited 'global-ignores' value being augmented by a non-inheritable 'svn:ignore' value should serve as a general model for how overriding should be done in an inherited properties system, that's a valid suggestion but it doesn't look like an elegant one. - Julian
Re: [RFC] Server Dictated Configuration
On 30.01.2012 17:05, Julian Foad wrote: No. The way I read the proposed 'server-dictated config' scheme, it didn't include a way to configure different values for 'global-ignores' to apply to different directories inside the WC, only for transmitting a single value of 'global-ignores' which could depend on the root directory of the WC. Huh? How does that make sense in, e.g., the ASF repository? -- Brane
Re: [RFC] Server Dictated Configuration
On Mon, Jan 30, 2012 at 11:05 AM, Julian Foad julianf...@btopenworld.com wrote: Branko Čibej wrote: On 27.01.2012 12:53, Julian Foad wrote: Branko Čibej wrote: On 27.01.2012 11:50, Julian Foad wrote: We need to see how we'd implement a reasonable system of svn:ignores and auto-props using the proposed inheritable properties system. The ability to see the inherited value and then merge in a child-defined value [...] is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. [...] No, you need to give the inherited properties that carry server-dictated configuration a different name, don't you think? Whether the merging is then done server-side or client-side is almost a bikeshed. I'm not quite sure what you mean. Could you give a specific example? [...] One way to achieve server-dictated configuration of ignores would be to make the server control the 'global-ignores' [config setting]. But for the purposes of exploring inheritable properties, let's ignore the 'global-ignores' config setting and assume that we're going to control the ignores through (inherited) properties alone. [...] Heh, but I fail to see a semantic difference between the two cases. :) An inherited properties design implies client-side setting of the inherited properties, whereas the design for server-dictated configuration implies that setting will be done server-side by an administrator. For either approach, I would ask: how would you go about setting up a useful hierarchy of ignore patterns? In the server-side case, you can say we'll just start with a simple config file format and defer that problem; somebody can design a more powerful config system for the administrator to use, later. So I asked specifically about how one would conveniently define ignore-patterns hierarchically in a generally useful inherited properties design. Since the server-dictated global-ignores would only apply to a certain subtree in the repository, it would /already/ behave as if it were an inherited svn:ignore property, and what's more, would be implicitly merged by existing client implementation with any svn:ignore properties that subtree happens to contain. No. The way I read the proposed 'server-dictated config' scheme, it didn't include a way to configure different values for 'global-ignores' to apply to different directories inside the WC, only for transmitting a single value of 'global-ignores' which could depend on the root directory of the WC. That is incorrect, the server dictated configuration proposal (http://wiki.apache.org/subversion/ServerDictatedConfiguration) supports different configuration values by path: [[[ Behavioral specification The high-level behavior for server-dictated configuration is relatively simple: the repository maintains a list of configuration parameters and values which, as necessary, the server provides to the client. The client, then, behaves in accordance with the server-dictated configuration. Subversion could recognize multiple levels of possible hierarchy in the server-side configuration: server-wide, per repository, or per repository-path. The current plan is to allow configuration at the most granular level, per repository-path. ]]] Paul But anyway, my point was to explore how useful the inherited properties idea would be in general, using ignore patterns as an example. If you're suggesting that this example of an inherited 'global-ignores' value being augmented by a non-inheritable 'svn:ignore' value should serve as a general model for how overriding should be done in an inherited properties system, that's a valid suggestion but it doesn't look like an elegant one. - Julian
Re: [RFC] Server Dictated Configuration
Paul Burba wrote: Julian Foad wrote: [...] The way I read the proposed 'server-dictated config' scheme, it didn't include a way to configure different values for 'global-ignores' to apply to different directories inside the WC, [...] That is incorrect, the server dictated configuration proposal (http://wiki.apache.org/subversion/ServerDictatedConfiguration) supports different configuration values by path: Oh, I'm sorry, I misremembered that. - Julian
Re: [RFC] Server Dictated Configuration
Hi all, First of all, thanks for working on server dictated config and inherited properties. We use Subversion as the core of a Document CMS (with focus on structured XML authoring). Some of the components we develop are available as open-source: http://repossearch.com/ We would be absolutely thrilled to see inherited properties implemented! There are many potential use cases for us, configuration of our services being one of the most imminent. Our primary input to this thread is that we would very much like inherited properties to be a generic feature accessible to other uses than core Subversion, preferably with API-support. On 28 jan 2012, at 10:30, Ivan Zhakov wrote: Are you suggesting that any property is inheritable simply by asking it to be so? What we are talking about here is a way to differentiate inheritable vs. non-inheritable properties (or as Johan put it: discern inheritable props from normal ones). Yes, that's exactly what I proposed. But I'm fine with idea differentiate properties behavior using dedicated namespaces (svn:i and svn:inheritable:). While following the thread, I have debated this with myself over the last few days and each approach has advantages. Consumer inheritance (the property consumer decides which are inheritable) + Less effort to implement + No difficulty defining what makes a property inheritable - Access control issues/regression. Personally, I would be ok with a regression along the lines that a user with access further down the tree is allowed to read properties on the parent directories (the user already knows about them from the URL). The only users objecting to that regression are those that store secret information in properties on directories (rare?). The bigger problem is that it is counter-intuitive behaviour to disclose those properties. - Performance and difficulty for the consumer (but more flexibe). Defined inheritance (the server decides) - More effort to implement. - Needs a definition of which properties are inheritable. I would suggest a server config (regex perhaps) that defaults to matching *:inherit:*, potentially limiting the first * to a single prefix (disallow ':'). If someone has used xyz:inherit:* the server can be configured more restrictively with svn:inherit:*. - Less flexible with regards to different merging/override behaviour when property is defined on multiple directories. + Defined, and fairly intuitive access control behaviour. + Possibility to provide better APIs I am fine with either way, but I am swaying towards defined inheritance mainly because of the last item, better APIs. Both the WC and the server could make it much simpler for the consumer to read and set inherited properties. Specifically, the server is in a much better position to efficiently (fewer round-trips) respond to proplist AND provide a single proplist containing both regular and inherited properties. /Thomas Å.
Re: [RFC] Server Dictated Configuration
On Mon, Jan 23, 2012 at 21:16, Paul Burba ptbu...@gmail.com wrote: On Sat, Jan 21, 2012 at 9:05 AM, Ivan Zhakov i...@visualsvn.com wrote: On Thu, Jan 19, 2012 at 03:23, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 5:02 PM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:55 PM, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:33 AM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 12:51 AM, Paul Burba ptbu...@gmail.com wrote: [...] We could chose to make things simple and follow the svn:mergeinfo model of inheritance: a) If a path has an explicit given property then it doesn't inherit that property; the explicit value is the complete value. b) If a path doesn't explicitly have a given property then it inherits that property from its nearest parent with the property explicitly set on it. No merging of inherited with explicit properties, so no conflicts, just keep it simple. I'm just thinking out loud here, but what about the following approach: - These new svn properties (e.g. all properties in de 'svn:conf:' namespace) are always *inheritable*. - But whether or not they are *inherited* (and in what way) is up to the sub-tree. 'In what way': I'm thinking about inheritance vs. extending/appending vs. override. Hi Johan, (The following may be the same as what you describe above, but just to be clear...) Let's keep in mind we (or at least I :-) have been talking about two closely related, but ultimately separate ideas: 1) Generic inherited properties 2) How to use inherited properties to facilitate repository-dictated auto-props and global-ignores. Agreed. Let's take repository dictated auto-props and global-ignores as an example of inherited properties in general. Viewed from the perspective of #1, a subtree either: a) Explicitly has a property set on it b) Doesn't have that explicit property but inherits it from a path-wise ancestor c) Doesn't have the the explicit property nor does it inherit it from any ancestor. Ok. But we can also dispense with c, as long as we're talking about inheritable props: we could say that they are always inherited, no matter what (except if you override them of course). I was being a bit pedantic in c). I only meant that one possible state is that the property in question is literally not set *anywhere*. Which makes me wonder: how will we discern inheritable props from normal ones? Merely saying that the svn:conf: namespace (for instance) means that it's always inheritable, will not cut it if we're talking about a generic feature ... Yeah, I've been thinking about this. As you say, the solution for our own inheritable properties is simple. Since Subversion already reserves properties beginning with svn: for its own use we could just extend it and say anything beginning with svn:inheritable: is inheritable. Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root). Hi Ivan, Are you suggesting that any property is inheritable simply by asking it to be so? What we are talking about here is a way to differentiate inheritable vs. non-inheritable properties (or as Johan put it: discern inheritable props from normal ones). Yes, that's exactly what I proposed. But I'm fine with idea differentiate properties behavior using dedicated namespaces (svn:i and svn:inheritable:). -- Ivan Zhakov
Re: [RFC] Server Dictated Configuration
Paul Burba wrote: See http://wiki.apache.org/subversion/InheritedProperties for what I'm envisioning. I've added some notes to the Wiki page. We need to see how we'd implement a reasonable system of svn:ignores and auto-props using the proposed inheritable properties system. The ability to see the inherited value and then merge in a child-defined value (adding/subtracting/overriding semantic sub-elements within the property value) is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. To do that, we will need APIs that can fetch the inherited value and the explicitly defined value for the same path as two separate values, so that the client code can combine them according to its own rules. However, an alternative is to provide the required ('ignores' for example) functionality in a way that matches the simple override semantics and does not use the semantics of the original 'svn:ignores' property. For example, instead of a multi-line value it could use a whole set of properties with one path pattern encoded into each property name: On ^/subversion/trunk: { svn:i:ignore:*.obj=True, svn:i:ignore:*.o=True } On ^/subversion/trunk/bindings: { svn:i:ignore:*.pyc=True, svn:i:ignore:*.o=False } The total set of (inherited and explicit) inheritable ignore rules is thus given by the total set of properties { svn:i:ignore-*.obj=True, svn:i:ignore:*.o=False, svn:i:ignore:*.pyc=True }. We could support both ways but I'm concerned that exposing both the inherited and explicit versions of each property value may make the API semantics more complex than necessary. - Julian Ivan Zhakov wrote: Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root).
Re: [RFC] Server Dictated Configuration
On 27.01.2012 11:50, Julian Foad wrote: Paul Burba wrote: See http://wiki.apache.org/subversion/InheritedProperties for what I'm envisioning. I've added some notes to the Wiki page. We need to see how we'd implement a reasonable system of svn:ignores and auto-props using the proposed inheritable properties system. The ability to see the inherited value and then merge in a child-defined value (adding/subtracting/overriding semantic sub-elements within the property value) is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. To do that, we will need APIs that can fetch the inherited value and the explicitly defined value for the same path as two separate values, so that the client code can combine them according to its own rules. No, you need to give the inherited properties that carry server-dictated configuration a different name, don't you think? Whether the merging is then done server-side or client-side is almost a bikeshed. -- Brane
Re: [RFC] Server Dictated Configuration
Branko Čibej wrote: On 27.01.2012 11:50, Julian Foad wrote: We need to see how we'd implement a reasonable system of svn:ignores and auto-props using the proposed inheritable properties system. The ability to see the inherited value and then merge in a child-defined value (adding/subtracting/overriding semantic sub-elements within the property value) is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. To do that, we will need APIs that can fetch the inherited value and the explicitly defined value for the same path as two separate values, so that the client code can combine them according to its own rules. No, you need to give the inherited properties that carry server-dictated configuration a different name, don't you think? Whether the merging is then done server-side or client-side is almost a bikeshed. I'm not quite sure what you mean. Could you give a specific example? I used the 'ignores' functionality in my example and was about to suggest you do the same, but there's a source of confusion: we currently have two completely different ways to specify ignores (client config 'global-ignores', and 'svn:ignore' property on a directory). One way to achieve server-dictated configuration of ignores would be to make the server control the 'global-ignores' setting in the client configuration, as in http://wiki.apache.org/subversion/ServerDictatedConfiguration. But for the purposes of exploring inheritable properties, let's ignore the 'global-ignores' config setting and assume that we're going to control the ignores through (inherited) properties alone. That's what I was assuming when I showed my { svn:i:ignore:*.pyc, ... } example. - Julian
Re: [RFC] Server Dictated Configuration
On Thu, Jan 26, 2012 at 5:22 PM, Daniel Shahaf danie...@elego.de wrote: Paul Burba wrote on Thu, Jan 26, 2012 at 16:55:48 -0500: On Tue, Jan 24, 2012 at 1:57 PM, C. Michael Pilato cmpil...@collab.net wrote: I must have blanked out through the sequence of emails that got us from let's solve a couple of oft-reported user issues regarding auto-props and ignores to let's implement custom inherited properties. But in general, I'm with Ivan here. Subversion's behavior is unaffected by user-defined properties, so we have zero obligation to implement any sort of advanced handling APIs that may or may not ever be used in conjunction with those properties. I'm not proposing anything like that... We need to provide a way to set properties; we need to provide a way to get them (as efficiently as possible). Agreed. Just the basics, that is all I am suggesting. The users can then do whatever they want with the results, choosing to interpret their own custom properties as inheritable or not based on criteria that is likewise custom Ah, here is where I suspect we have a disconnect. You say users can choose to interpret their own custom properties as inheritable. This seemingly implies that there is no real difference between normal user properties and inheritable user properties; it's all in how we chose to interpret them. Is this what you are saying? Because if it is, I think this is a mistake, as I've explained elsethread. I'll take another stab at explaining my objection: [It should be possible to read inheritable properties, but not other properties, from the repository root even without read access to it.] Hi Daniel, Sure, the parent path might be the root of the repository, but not necessarily. What you are stating above is only a specific example of what I mean. I'm arguing for a more general principle: If you have read access to a path, then you can read the inheritable properties of that path's parents, regardless of what access you have to the parents. Make sense? Why can't the admin simply set the property on every project root? Well we could, but what if a user doesn't have access to the project root? We could step down to a deeper subtree level and set the inheritable property. But how deep? To be sure the inheritable property applies to all users, we'd need to consult the authz configuration to be sure no user/path combination can't inherit the property. That's not very convenient, so we could instead... Just like we, redundantly, set svn:ignore to the same value on all subversion/tests/libsvn_*/ directories. ...just set our property on *every* directory in the subtree we want the inheritable property to apply to. But is that really an inheritable property? Maybe in a very loose sense, but it doesn't seem very useful. Paul Daniel P.S. Sorry if my []-summarizing of your email is inaccurate.
Re: [RFC] Server Dictated Configuration
On 27.01.2012 12:53, Julian Foad wrote: Branko Čibej wrote: On 27.01.2012 11:50, Julian Foad wrote: We need to see how we'd implement a reasonable system of svn:ignores and auto-props using the proposed inheritable properties system. The ability to see the inherited value and then merge in a child-defined value (adding/subtracting/overriding semantic sub-elements within the property value) is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. To do that, we will need APIs that can fetch the inherited value and the explicitly defined value for the same path as two separate values, so that the client code can combine them according to its own rules. No, you need to give the inherited properties that carry server-dictated configuration a different name, don't you think? Whether the merging is then done server-side or client-side is almost a bikeshed. I'm not quite sure what you mean. Could you give a specific example? I used the 'ignores' functionality in my example and was about to suggest you do the same, but there's a source of confusion: we currently have two completely different ways to specify ignores (client config 'global-ignores', and 'svn:ignore' property on a directory). One way to achieve server-dictated configuration of ignores would be to make the server control the 'global-ignores' setting in the client configuration, as in http://wiki.apache.org/subversion/ServerDictatedConfiguration. But for the purposes of exploring inheritable properties, let's ignore the 'global-ignores' config setting and assume that we're going to control the ignores through (inherited) properties alone. That's what I was assuming when I showed my { svn:i:ignore:*.pyc, ... } example. Heh, but I fail to see a semantic difference between the two cases. :) Since the server-dictated global-ignores would only apply to a certain subtree in the repository, it would /already/ behave as if it were an inherited svn:ignore property, and what's more, would be implicitly merged by existing client implementation with any svn:ignore properties that subtree happens to contain. So then, if someone wants to have different inherited global ignores, deeper down, one can add them. Of course this doesn't quite cover all the use cases, specifically the one where you want to just extend any existing global ignores, which your svn:i: namespace seems to provide. However, when you say automatically merge, remember that merge can be additive or subtractive ... so ... your svn:i: still doesn't cover all the use cases, you'd have to, e.g., introduce svn:add:ignore and svn:del:ignore. At which point I begin to wonder if this part is even worth the extra complexity. -- Brane
Re: [RFC] Server Dictated Configuration
On Fri, Jan 27, 2012 at 5:50 AM, Julian Foad julianf...@btopenworld.com wrote: Paul Burba wrote: See http://wiki.apache.org/subversion/InheritedProperties for what I'm envisioning. I've added some notes to the Wiki page. We need to see how we'd implement a reasonable system of svn:ignores and auto-props using the proposed inheritable properties system. Hi Julian, I'd like the inherited properties proposal to stand on it's own, but since my first goal is to implement inheritable 'autoprops' and 'ignores' properties, I suppose it's fair enough to ask how it would support those. The ability to see the inherited value and then merge in a child-defined value (adding/subtracting/overriding semantic sub-elements within the property value) is essential if we're going to implement these features using properties with semantics like the existing 'svn:ignores'. Why do we need to subtract and override? Why not stick with what we have today, is there a strong user demand for more? What I mean is...well let's back up first and look at what we have today: 1) The svn:ignore property set on directories and effecting only that directory's immediate children. 2) The [miscellany] global-ignores configuration which affects the entire working copy. 3) Right now (assuming a WC operation) the value of the svn:ignore property (if any) on a path's parent directory is simply appended to the global-ignores runtime config value to give the final answer on what to ignore. Simple concatenation. Now let's say we implement inheritable properties as I described in the wiki and want to use an inheritable property to supplement the existing mechanisms for svn:ignores/global-ignores. Isn't that as simple as this? 4) We add a new reserved inheritable property svn:i:ignores which has the exact same format as svn:ignore. 5) Again assuming a WC operation, we take a path's inherited (or explicit) svn:i:ignores property value, the svn:ignore property (if any) on a path's parent directory, and the global-ignores runtime config value and append all three together to get the final answer on what to ignore. I take it you view this as insufficient? To do that, we will need APIs that can fetch the inherited value and the explicitly defined value for the same path as two separate values, so that the client code can combine them according to its own rules. As Brane points out, simple using a different namespace for the inheritable properties will accomplish this. No need to introduce the concept of a path having both an explicit property 'X=VAL' while at the same time inheriting property 'X=PARENT_VAL'. Paul However, an alternative is to provide the required ('ignores' for example) functionality in a way that matches the simple override semantics and does not use the semantics of the original 'svn:ignores' property. For example, instead of a multi-line value it could use a whole set of properties with one path pattern encoded into each property name: On ^/subversion/trunk: { svn:i:ignore:*.obj=True, svn:i:ignore:*.o=True } On ^/subversion/trunk/bindings: { svn:i:ignore:*.pyc=True, svn:i:ignore:*.o=False } The total set of (inherited and explicit) inheritable ignore rules is thus given by the total set of properties { svn:i:ignore-*.obj=True, svn:i:ignore:*.o=False, svn:i:ignore:*.pyc=True }. We could support both ways but I'm concerned that exposing both the inherited and explicit versions of each property value may make the API semantics more complex than necessary. - Julian Ivan Zhakov wrote: Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root).
Re: [RFC] Server Dictated Configuration
On Tue, Jan 24, 2012 at 1:57 PM, C. Michael Pilato cmpil...@collab.net wrote: On 01/21/2012 09:05 AM, Ivan Zhakov wrote: On Thu, Jan 19, 2012 at 03:23, Paul Burba ptbu...@gmail.com wrote: Yeah, I've been thinking about this. As you say, the solution for our own inheritable properties is simple. Since Subversion already reserves properties beginning with svn: for its own use we could just extend it and say anything beginning with svn:inheritable: is inheritable. Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root). I must have blanked out through the sequence of emails that got us from let's solve a couple of oft-reported user issues regarding auto-props and ignores to let's implement custom inherited properties. But in general, I'm with Ivan here. Subversion's behavior is unaffected by user-defined properties, so we have zero obligation to implement any sort of advanced handling APIs that may or may not ever be used in conjunction with those properties. I'm not proposing anything like that... We need to provide a way to set properties; we need to provide a way to get them (as efficiently as possible). Agreed. Just the basics, that is all I am suggesting. The users can then do whatever they want with the results, choosing to interpret their own custom properties as inheritable or not based on criteria that is likewise custom Ah, here is where I suspect we have a disconnect. You say users can choose to interpret their own custom properties as inheritable. This seemingly implies that there is no real difference between normal user properties and inheritable user properties; it's all in how we chose to interpret them. Is this what you are saying? Because if it is, I think this is a mistake, as I've explained elsethread. I'll take another stab at explaining my objection: 1) First, the obvious: Inheritable properties set on a path should be inherited by all the paths underneath it (assuming none of those subtrees have the same property set). 2) If we accept that #1 is a worthy goal, then it follows that we only need set an inheritable property on the common root of all paths we want that property to apply to. Take two likely use cases: a) A repository administrator has a property they want to set on the whole repository, they should be able to simply set it on the root of the repository and be done with it. b) Given a repository like the ASF's, there are scores of projects rooted just below the repository root. Each project is largely managed by different groups that likely have different needs. These different needs likely mean that different inheritable properties apply to different projects. Like 'a' above, the project owners should be able to set the inheritable properties that apply to their project on the root path of their project and be done with it. Seem reasonable so far? 3) To make #2 feasible, inheritable properties should behave like svn:mergeinfo, they should always be inheritable from a parent path, even if you only have access to the child path. For example, say we have a repository structured like so: / /projX /projX/trunk ... /projX/branches /projX/branches/1.0 ... /projX/branches/1.1 ... User Regular_Joe has rw access to /projX/trunk' and '/projX/branches', but no access to the repos root or '/projX' proper. However, '/' has numerous repository-wide inheritable properties set on it and '/projX' likewise has several project-wide inheritable properties. We face two choices: a) Either Regular_Joe can know the property values of the inheritable properties on '/' and '/projX', despite not having any access to those paths (did I already mention that this is how svn:mergeinfo has behaved since 1.5? :-) *OR* b) Regular_Joe can't know the property values inherited from paths he doesn't have access to. This means that whoever sets up inheritable properties must take the repository's authz settings into account if they want to be sure the property really applies to all the paths they want it to apply to. I hope I don't need to go into detail as to why this would be both cumbersome and mistake prone to setup and maintain. 4) Ok, let's assume for a moment that '3a' is the right approach, if we have access to a path we can know what properties that path inherits regardless of our access to the path's parents. But if we don't differentiate between 'normal' and 'inheritable' properties, then we can know
Re: [RFC] Server Dictated Configuration
Paul Burba wrote on Thu, Jan 26, 2012 at 16:55:48 -0500: On Tue, Jan 24, 2012 at 1:57 PM, C. Michael Pilato cmpil...@collab.net wrote: I must have blanked out through the sequence of emails that got us from let's solve a couple of oft-reported user issues regarding auto-props and ignores to let's implement custom inherited properties. But in general, I'm with Ivan here. Subversion's behavior is unaffected by user-defined properties, so we have zero obligation to implement any sort of advanced handling APIs that may or may not ever be used in conjunction with those properties. I'm not proposing anything like that... We need to provide a way to set properties; we need to provide a way to get them (as efficiently as possible). Agreed. Just the basics, that is all I am suggesting. The users can then do whatever they want with the results, choosing to interpret their own custom properties as inheritable or not based on criteria that is likewise custom Ah, here is where I suspect we have a disconnect. You say users can choose to interpret their own custom properties as inheritable. This seemingly implies that there is no real difference between normal user properties and inheritable user properties; it's all in how we chose to interpret them. Is this what you are saying? Because if it is, I think this is a mistake, as I've explained elsethread. I'll take another stab at explaining my objection: [It should be possible to read inheritable properties, but not other properties, from the repository root even without read access to it.] Make sense? Why can't the admin simply set the property on every project root? Just like we, redundantly, set svn:ignore to the same value on all subversion/tests/libsvn_*/ directories. Daniel P.S. Sorry if my []-summarizing of your email is inaccurate.
Re: [RFC] Server Dictated Configuration
On 01/21/2012 09:05 AM, Ivan Zhakov wrote: On Thu, Jan 19, 2012 at 03:23, Paul Burba ptbu...@gmail.com wrote: Yeah, I've been thinking about this. As you say, the solution for our own inheritable properties is simple. Since Subversion already reserves properties beginning with svn: for its own use we could just extend it and say anything beginning with svn:inheritable: is inheritable. Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root). I must have blanked out through the sequence of emails that got us from let's solve a couple of oft-reported user issues regarding auto-props and ignores to let's implement custom inherited properties. But in general, I'm with Ivan here. Subversion's behavior is unaffected by user-defined properties, so we have zero obligation to implement any sort of advanced handling APIs that may or may not ever be used in conjunction with those properties. We need to provide a way to set properties; we need to provide a way to get them (as efficiently as possible). The users can then do whatever they want with the results, choosing to interpret their own custom properties as inheritable or not based on criteria that is likewise custom, and performing the requisite calculations for merging such properties using equally custom code around our rather sufficient APIs. Later, if sufficient user-generated demand -- based on real-world usage scenarios -- for consistent handling/merging of properties which are, by convention, deemed inheritable arise, we can explore adding public APIs to consistify this handling/merging. But please, let's not overdesign this for the sake of what if. -- C. Michael Pilato cmpil...@collab.net CollabNet www.collab.net Distributed Development On Demand signature.asc Description: OpenPGP digital signature
Re: [RFC] Server Dictated Configuration
On 01/21/2012 09:05 AM, Ivan Zhakov wrote: On Thu, Jan 19, 2012 at 03:23, Paul Burba ptbu...@gmail.com wrote: Yeah, I've been thinking about this. As you say, the solution for our own inheritable properties is simple. Since Subversion already reserves properties beginning with svn: for its own use we could just extend it and say anything beginning with svn:inheritable: is inheritable. Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root). I must have blanked out through the sequence of emails that got us from let's solve a couple of oft-reported user issues regarding auto-props and ignores to let's implement custom inherited properties. But in general, I'm with Ivan here. Subversion's behavior is unaffected by user-defined properties, so we have zero obligation to implement any sort of advanced handling APIs that may or may not ever be used in conjunction with those properties. We need to provide a way to set properties; we need to provide a way to get them (as efficiently as possible). The users can then do whatever they want with the results, choosing to interpret their own custom properties as inheritable or not based on criteria that is likewise custom, and performing the requisite calculations for merging such properties using equally custom code around our rather sufficient APIs. Later, if sufficient user-generated demand arises -- based on real-world usage scenarios -- for consistent handling/merging of properties which are, by convention, deemed inheritable, *then* we can explore adding public APIs and such to consistify this handling/merging. But please, let's not overdesign this for the sake of what if. -- C. Michael Pilato cmpil...@collab.net CollabNet www.collab.net Distributed Development On Demand signature.asc Description: OpenPGP digital signature
Re: [RFC] Server Dictated Configuration
On Sat, Jan 21, 2012 at 9:05 AM, Ivan Zhakov i...@visualsvn.com wrote: On Thu, Jan 19, 2012 at 03:23, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 5:02 PM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:55 PM, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:33 AM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 12:51 AM, Paul Burba ptbu...@gmail.com wrote: [...] We could chose to make things simple and follow the svn:mergeinfo model of inheritance: a) If a path has an explicit given property then it doesn't inherit that property; the explicit value is the complete value. b) If a path doesn't explicitly have a given property then it inherits that property from its nearest parent with the property explicitly set on it. No merging of inherited with explicit properties, so no conflicts, just keep it simple. I'm just thinking out loud here, but what about the following approach: - These new svn properties (e.g. all properties in de 'svn:conf:' namespace) are always *inheritable*. - But whether or not they are *inherited* (and in what way) is up to the sub-tree. 'In what way': I'm thinking about inheritance vs. extending/appending vs. override. Hi Johan, (The following may be the same as what you describe above, but just to be clear...) Let's keep in mind we (or at least I :-) have been talking about two closely related, but ultimately separate ideas: 1) Generic inherited properties 2) How to use inherited properties to facilitate repository-dictated auto-props and global-ignores. Agreed. Let's take repository dictated auto-props and global-ignores as an example of inherited properties in general. Viewed from the perspective of #1, a subtree either: a) Explicitly has a property set on it b) Doesn't have that explicit property but inherits it from a path-wise ancestor c) Doesn't have the the explicit property nor does it inherit it from any ancestor. Ok. But we can also dispense with c, as long as we're talking about inheritable props: we could say that they are always inherited, no matter what (except if you override them of course). I was being a bit pedantic in c). I only meant that one possible state is that the property in question is literally not set *anywhere*. Which makes me wonder: how will we discern inheritable props from normal ones? Merely saying that the svn:conf: namespace (for instance) means that it's always inheritable, will not cut it if we're talking about a generic feature ... Yeah, I've been thinking about this. As you say, the solution for our own inheritable properties is simple. Since Subversion already reserves properties beginning with svn: for its own use we could just extend it and say anything beginning with svn:inheritable: is inheritable. Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root). Hi Ivan, Are you suggesting that any property is inheritable simply by asking it to be so? What we are talking about here is a way to differentiate inheritable vs. non-inheritable properties (or as Johan put it: discern inheritable props from normal ones). It sounds to me as if you are proposing there be no such concept of inheritable vs. non-inheritable properties. If that is the case, then *all* properties are potentially inheritable, not just those that were *intentionally* created as inheritable. The problem with this is that users could then see what properties exist on paths they don't have read access to. I've made my argument earlier in this thread as to why I think *inheritable* properties set on an unreadable parent should be inherited by a readable child, but this logic doesn't extend to any arbitrary property. Paul Ivan Zhakov
Re: [RFC] Server Dictated Configuration
On Sat, Jan 21, 2012 at 2:11 PM, Branko Čibej br...@apache.org wrote: On 17.01.2012 02:28, Hyrum K Wright wrote: ... and something like inheritable props my fit in that model, though I'd had to make the feature dependency tree another level deeper. Before you jump on the inheritable properties bandwagon, consider that server-mandated configuration is not inherently versioned in the same way as ordinary node properties (nor, for that matter, are ACLs). There is a valid case for allowing modification of server-side configuration (and/or ACLs) for existing, historical revisions, which you cannot do with node properties today. So these attributes behave like revprops in the sense that you can change them for an historical revision, but like node properties in the sense that they apply to a path and/or subtree. It's a different-but-parallel structure to node properties. You still want to maintain an audit trail of historic modifications, however; so that you can ask, e.g., what was the effective ACL of ^/foo/bar two weeks ago and who changed it since then. This trail is, by the way, something we don't provide for revprops, but should IMO. You could say that revprops are just a special case of this attribute that are constrained to the repository root. Hi Brane, Valid points. An audit trail of configuration changes (particularly those that have a direct effect on the repository's contents, e.g. auto-props, global-ignores), is one of the advantages to server-dictated-config-via-inheritable-versioned-props approach over the one in http://wiki.apache.org/subversion/ServerDictatedConfiguration. And certainly a versioned revprop mechanism would have its uses. However I suspect that much of the former can be satisfactorily achieved without the latter...though I'm not 100% convinced by any proposed solution just yet[1]. Right now I'm working on a new wiki detailing a possible approach to generic inherited properties, separate from the question of server dictated configuration. If I'm going to bother with inheritable versioned properties to address some/all of server dictated config, then I want I want these inheritable properties to be generally useful (as opposed to our current inheritable property, svn:mergeinfo, which is obviously quite specialized and has no use outside of merge-tracking). Anyhow, I suppose all of the above is just a long-winded way of saying I'll keep your points in mind! Paul [1] I only have one foot on the inheritable properties bandwagon :-) -- Brane
Re: [RFC] Server Dictated Configuration
On Thu, Jan 19, 2012 at 03:23, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 5:02 PM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:55 PM, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:33 AM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 12:51 AM, Paul Burba ptbu...@gmail.com wrote: [...] We could chose to make things simple and follow the svn:mergeinfo model of inheritance: a) If a path has an explicit given property then it doesn't inherit that property; the explicit value is the complete value. b) If a path doesn't explicitly have a given property then it inherits that property from its nearest parent with the property explicitly set on it. No merging of inherited with explicit properties, so no conflicts, just keep it simple. I'm just thinking out loud here, but what about the following approach: - These new svn properties (e.g. all properties in de 'svn:conf:' namespace) are always *inheritable*. - But whether or not they are *inherited* (and in what way) is up to the sub-tree. 'In what way': I'm thinking about inheritance vs. extending/appending vs. override. Hi Johan, (The following may be the same as what you describe above, but just to be clear...) Let's keep in mind we (or at least I :-) have been talking about two closely related, but ultimately separate ideas: 1) Generic inherited properties 2) How to use inherited properties to facilitate repository-dictated auto-props and global-ignores. Agreed. Let's take repository dictated auto-props and global-ignores as an example of inherited properties in general. Viewed from the perspective of #1, a subtree either: a) Explicitly has a property set on it b) Doesn't have that explicit property but inherits it from a path-wise ancestor c) Doesn't have the the explicit property nor does it inherit it from any ancestor. Ok. But we can also dispense with c, as long as we're talking about inheritable props: we could say that they are always inherited, no matter what (except if you override them of course). I was being a bit pedantic in c). I only meant that one possible state is that the property in question is literally not set *anywhere*. Which makes me wonder: how will we discern inheritable props from normal ones? Merely saying that the svn:conf: namespace (for instance) means that it's always inheritable, will not cut it if we're talking about a generic feature ... Yeah, I've been thinking about this. As you say, the solution for our own inheritable properties is simple. Since Subversion already reserves properties beginning with svn: for its own use we could just extend it and say anything beginning with svn:inheritable: is inheritable. Another option is make caller to know how to handle properties. I.e. introduce call like svn_get_inhertiable_prop(wc_path, propname) returning chain of configured properties with repos_path starting from given wc_path. For example call to svn_get_inheritable_prop(wcroot/foo/bar, property) return: /repos/project/trunk/foo - value1 /repos/project/trunk - value2 /repos/ - value3 Then caller may implement any logic to merge them. At implementation level we can store all properties and WC root (and for each switched root). -- Ivan Zhakov
Re: [RFC] Server Dictated Configuration
On 17.01.2012 02:28, Hyrum K Wright wrote: ... and something like inheritable props my fit in that model, though I'd had to make the feature dependency tree another level deeper. Before you jump on the inheritable properties bandwagon, consider that server-mandated configuration is not inherently versioned in the same way as ordinary node properties (nor, for that matter, are ACLs). There is a valid case for allowing modification of server-side configuration (and/or ACLs) for existing, historical revisions, which you cannot do with node properties today. So these attributes behave like revprops in the sense that you can change them for an historical revision, but like node properties in the sense that they apply to a path and/or subtree. It's a different-but-parallel structure to node properties. You still want to maintain an audit trail of historic modifications, however; so that you can ask, e.g., what was the effective ACL of ^/foo/bar two weeks ago and who changed it since then. This trail is, by the way, something we don't provide for revprops, but should IMO. You could say that revprops are just a special case of this attribute that are constrained to the repository root. -- Brane
Re: [RFC] Server Dictated Configuration
On 17.01.2012 20:54, Johan Corveleyn wrote: But, but ... if you're able to checkout ^/foo/bar/baz, then you already know that foo and foo/bar In the ACL world, there is a difference between lookup and read access. In your example, the user has permission to lookup ^/foo and ^/foo/bar, but not to read them; whereas she can read ^/foo/bar/baz. Lookup implies that you can perform operations on the node's descendants (based on their access flags), but not access the node's properties -- and that includes not allowing directory enumeration. In other words, lookup access on ^/foo/bar means that you may be able to open(^/foo/bar/baz) if you have appropriate access to .../baz, but not readdir(^/foo/bar) == so someone has to tell you that ^/foo/bar/baz exists since you can't discover that by walking down the directory tree. Some models always allow lookup, others allow you to turn it off. It all depends on how many non-obvious edge cases you want to introduce in your ACL model. :) -- Brane
Re: [RFC] Server Dictated Configuration
On Tue, Jan 17, 2012 at 11:36 AM, Paul Burba ptbu...@gmail.com wrote: On Mon, Jan 16, 2012 at 8:28 PM, Hyrum K Wright hyrum.wri...@wandisco.com wrote: On Mon, Jan 16, 2012 at 5:51 PM, Paul Burba ptbu...@gmail.com wrote: ... On Thu, Jan 5, 2012 at 4:52 PM, Hyrum K Wright hyrum.wri...@wandisco.com wrote: As I recall, there were a few reasons why inherited properties haven't been implemented. One is the client-side storage and lookup, which wc-ng has helped with. The other is what to do with non-checked out parent directories, which you mention above. Another problem is the various authz issues, similar to the infamous issue 3242 problems we had with copy and move. Maybe we can make a special case for inheritable properties, simply state that by definition, if you set an inheritable property on a path, then users, even those who don't have access to the path, but do have access to the path's children, can inherit the property. For example if a user has access to foo/bar/baz, then that user can inherit properties from foo, even if he doesn't have access to foo itself. All the repository has to tell the client is, Path foo/bar/baz inherted property NAME=VAL, it doesn't even need to say where it came from. If we don't make this exception then it seems to me that inheritable properties are dead in the water, at least as far as being a useful solution to server dictated auto-props and global-ignores. After having gone through the wc-ng experience, alarm bells start ringing every time somebody says we can make a special case for Hi Hyrum, I might have needlessly muddied the waters when I said special case. To be clear, I meant that *all* inheritable properties (not just svn:auto-props and svn:global-ignores) would behave this way in the *general* case. The special case is not about svn:auto-props and svn:global-ignores in particular, but rather that we can know some information (i.e. the inherited property value) about a path-wise ancestor we don't have access to. Please, please, please consider if that's really needed, of if a more generalizable solution is appropriate. Well if that is the ability of a path (which we have read access to) to know the inherited value of a property from the path's (path-wise) ancestor (which we have no access to), then yes, I think it is really needed. Why? Because without this behavior inheritable properties are crippled from the start. I should also point out that the proposed behavior is exactly how our only inheritable property to date (i.e. svn:mergeinfo) has been treated since 1.5. That is, you can inherit mergeinfo from a path you have no access to, as long as you have access to the path doing the inheriting. Paul
Re: [RFC] Server Dictated Configuration
On Tue, Jan 17, 2012 at 5:02 PM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:55 PM, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:33 AM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 12:51 AM, Paul Burba ptbu...@gmail.com wrote: [...] We could chose to make things simple and follow the svn:mergeinfo model of inheritance: a) If a path has an explicit given property then it doesn't inherit that property; the explicit value is the complete value. b) If a path doesn't explicitly have a given property then it inherits that property from its nearest parent with the property explicitly set on it. No merging of inherited with explicit properties, so no conflicts, just keep it simple. I'm just thinking out loud here, but what about the following approach: - These new svn properties (e.g. all properties in de 'svn:conf:' namespace) are always *inheritable*. - But whether or not they are *inherited* (and in what way) is up to the sub-tree. 'In what way': I'm thinking about inheritance vs. extending/appending vs. override. Hi Johan, (The following may be the same as what you describe above, but just to be clear...) Let's keep in mind we (or at least I :-) have been talking about two closely related, but ultimately separate ideas: 1) Generic inherited properties 2) How to use inherited properties to facilitate repository-dictated auto-props and global-ignores. Agreed. Let's take repository dictated auto-props and global-ignores as an example of inherited properties in general. Viewed from the perspective of #1, a subtree either: a) Explicitly has a property set on it b) Doesn't have that explicit property but inherits it from a path-wise ancestor c) Doesn't have the the explicit property nor does it inherit it from any ancestor. Ok. But we can also dispense with c, as long as we're talking about inheritable props: we could say that they are always inherited, no matter what (except if you override them of course). I was being a bit pedantic in c). I only meant that one possible state is that the property in question is literally not set *anywhere*. Which makes me wonder: how will we discern inheritable props from normal ones? Merely saying that the svn:conf: namespace (for instance) means that it's always inheritable, will not cut it if we're talking about a generic feature ... Yeah, I've been thinking about this. As you say, the solution for our own inheritable properties is simple. Since Subversion already reserves properties beginning with svn: for its own use we could just extend it and say anything beginning with svn:inheritable: is inheritable. But for custom user properties? Ugh, yet again I see why this feature has been avoided :-) Either we declare that customer user properties names starting with some arbitrary string (inherited:?) will be treated as inherited or we require that inheritable properties be explicitly created as inheritable (i.e. some new option to propset and propedit). The latter of course means we'll have to make changes to the backends and wcng to store this new trait. The former means that we might make existing custom properties that share our chosen prefix into inheritable properties... ...Or maybe we invite users into our svn: namespace. Let them use the svn:inheritable: prefix while we reserve the svn:i: prefix for our own use. Seems like that would work. ~ I've also been pondering a closely related issue, namely how to differentiate between an explicitly set property and an inherited one. What I mean is this, say we have this working copy: branch-wc/ branch-wc/src/ branch-wc/src/main.c Further, let's suppose the inheritable property svn:inherit:auto-props, with the value *.c=svn:eol-style=native, is set on branch-wc/src. If we ask what properties are on branch-wc/src the output is obvious: svn pl -v src Properties on 'src': svn:inherit:auto-props *.c=svn:eol-style=native Now if we ask what properties are found on src\main.c, we all (I hope!) expect to see both the properties explicitly set on main.c, as well as those inherited from src: svn pl -v src\main.c Properties on 'src\main.c': svn:eol-style native svn:inherit:auto-props *.c=svn:eol-style=native But how does a user differentiate between an explicit and inherited property? We'll need to flag it somehow. For proplist this is relatively easy: svn pl -v src\main.c Properties on 'src\main.c': svn:eol-style native Inherited properties on 'src\main.c': svn:inherit:auto-props *.c=svn:eol-style=native svn pl src\main.c Properties on 'src\main.c': svn:eol-style Inherited properties on 'src\main.c': svn:inherit:auto-props But for propget it's a bit more difficult: svn pg svn:inherit:auto-props src *.c=svn:eol-style=native svn pg svn:inherit:auto-props src\main.c *.c=svn:eol-style=native Also, how
Re: [RFC] Server Dictated Configuration
On Tue, Jan 17, 2012 at 12:51 AM, Paul Burba ptbu...@gmail.com wrote: [...] We could chose to make things simple and follow the svn:mergeinfo model of inheritance: a) If a path has an explicit given property then it doesn't inherit that property; the explicit value is the complete value. b) If a path doesn't explicitly have a given property then it inherits that property from its nearest parent with the property explicitly set on it. No merging of inherited with explicit properties, so no conflicts, just keep it simple. I'm just thinking out loud here, but what about the following approach: - These new svn properties (e.g. all properties in de 'svn:conf:' namespace) are always *inheritable*. - But whether or not they are *inherited* (and in what way) is up to the sub-tree. 'In what way': I'm thinking about inheritance vs. extending/appending vs. override. I'm thinking a bit along the lines of OO-languages: some special syntax or keyword in the subtree's property signals that the property must be inherited. Let's say we use '$super' to point to the inherited prop, and if the prop is not specified, '=$super' is implied, then we could do: ^/ - svn:conf:ignore = *.o *.lo *.la *.al .libs ^/native-app - no prop needed ($super is implied)# inherit ^/native-app/python-mods - svn:conf:ignore = $super *.pyc# append ^/java-lib - svn:conf:ignore = *.class# override ^/java-lib/python-mods - svn:conf:ignore = $super *.pyc# append Or something like that. The situation would be more complex for 'auto-props', depending on how we choose to model that: as a single multiline property (svn:conf:auto-props), or as a set of properties, one for each 'pattern' ('svn:conf:auto-props:*.c = svn:eol-style=native', 'svn:conf:auto-props:*.java = svn:eol-style=native', ...). The latter looks more like how it's currently structured in client-side config. In the multiline case, we might like to make it possible to inherit everything ($super), but also to specify inheritance of individual 'patterns' out of the multiline prop (something like 'svn:conf:auto-props = *.c = $super(*.c) \n*.java = ...'). Hmmm, that gets complicated ... I'll stop here with thinking out loud :-). Essentially the same approach for svn:auto-props and svn:global-ignores, we do the same thing the current hierarchy of configuration does today: There is no merging of lower priority configuration, it's simply a matter of choosing the option value from the highest priority level the option is specified. Which would be, from highest to lowest priority: Command-line options* Explicit/inherited svn:auto-props/svn:global-ignores Per-user runtime configuration (${HOME}/.subversion/*) The per-user Registry values (Windows Only) Per-machine runtime configuration (/etc/subversion/*) The system-wide Registry values (Windows Only) * Yes Johan, your arguments for making CL options have the highest priority did sway me :-) On Fri, Jan 6, 2012 at 7:16 AM, Johan Corveleyn jcor...@gmail.com wrote: Come to think of it: using properties has the nice advantage that these configurations come along when a project is branched or tagged. Which is what most users would expect, I guess. Except when they don't expect it... The configurations would certainly come along if the path on which they were explicitly set was copied (just like any other versioned prop), but I don't think inherited properties should be copied, because then if the inherited properties on the destination differ from the copy source, then we'll need...wait for it...explicit subtree inherited properties on the destination (i.e. explicit properties that appear, seemingly from nowhere, on the copy destination). Properties popping into existence has proved quite confusing with svn:mergeinfo, where it is regrettably necessary in some cases, but for generic inherited properties do we really need this behavior? Why not simply state that inheritable properties are handled *exactly* like any other versioned property w.r.t. copies and moves, so users should plan accordingly. I don't mean to imply that there won't be cases where we might prefer to have inherited properties copied, it's just that neither approach is perfect for every use case, so why not go with the behavior that is simpler to implement, simpler to explain, and doesn't sprout new properties out of the ether? I agree we shouldn't be doing any special copying of inherited props that come from higher up the hierarchy than the branch-root. But I think that's not needed, if we clearly state that we limit the property inheritance to only path-wise hierarchy (not historical ancestry). So if a property is inherited because it's set on a path that's higher up the hierarchy than the branch-root, it will still apply to any new branches that are part of that same '(grand-)parent-path'. Properties set on ^/subversion will apply to trunk, as
Re: [RFC] Server Dictated Configuration
On Mon, Jan 16, 2012 at 8:28 PM, Hyrum K Wright hyrum.wri...@wandisco.com wrote: On Mon, Jan 16, 2012 at 5:51 PM, Paul Burba ptbu...@gmail.com wrote: ... On Thu, Jan 5, 2012 at 4:52 PM, Hyrum K Wright hyrum.wri...@wandisco.com wrote: As I recall, there were a few reasons why inherited properties haven't been implemented. One is the client-side storage and lookup, which wc-ng has helped with. The other is what to do with non-checked out parent directories, which you mention above. Another problem is the various authz issues, similar to the infamous issue 3242 problems we had with copy and move. Maybe we can make a special case for inheritable properties, simply state that by definition, if you set an inheritable property on a path, then users, even those who don't have access to the path, but do have access to the path's children, can inherit the property. For example if a user has access to foo/bar/baz, then that user can inherit properties from foo, even if he doesn't have access to foo itself. All the repository has to tell the client is, Path foo/bar/baz inherted property NAME=VAL, it doesn't even need to say where it came from. If we don't make this exception then it seems to me that inheritable properties are dead in the water, at least as far as being a useful solution to server dictated auto-props and global-ignores. After having gone through the wc-ng experience, alarm bells start ringing every time somebody says we can make a special case for Hi Hyrum, I might have needlessly muddied the waters when I said special case. To be clear, I meant that *all* inheritable properties (not just svn:auto-props and svn:global-ignores) would behave this way in the *general* case. The special case is not about svn:auto-props and svn:global-ignores in particular, but rather that we can know some information (i.e. the inherited property value) about a path-wise ancestor we don't have access to. Please, please, please consider if that's really needed, of if a more generalizable solution is appropriate. Well if that is the ability of a path (which we have read access to) to know the inherited value of a property from the path's (path-wise) ancestor (which we have no access to), then yes, I think it is really needed. Why? Because without this behavior inheritable properties are crippled from the start. Let's forget about auto-props and global-ignores for a minute and take a really simple generic use-case for inheritable properties: We want the inheritable property svn:inheritable-foo to apply to our entire repository. Ok, let's set it on the root of the repository. Sit back smugly, thinking, Ha! That was easy! Five minutes later: Oops, we have fine grained path-based access set up, not all users have access to the repository root so they won't inherit this property. We'll need to determine the subtrees within the repository that each unique user access configuration can read and then set svn:inheritable-foo on the roots of those subtrees. And we need to make sure we keep this in sync with any changes the access rules. This is an administrative nightmare! (And maybe what we're talking about here is a generalized solution for inheritable props. :) ) Sorry, I should have clearly stated from the get-go: I am talking about a *generalized* solution for inheritable properties, I'm just doing it with an eye to how it can be used to implement repository dictated autoprops/ignores. The latter is what I want to solve. The wiki proposed a solution and several people pointed out that the feature could be implemented with properties, so I'm investigating that option. Another thing to note is that there have been some rumblings about authz improvements, along the lines of an additional permission to say you can know about this directory. I know C-Mike has been thinking about this off-and-on since the 3242 debacle, and something like inheritable props my fit in that model, though I'd had to make the feature dependency tree another level deeper. Even if that was implemented today it's still an administrative nightmare, albeit a lesser one. In the example above we'd need to give the new special permission to the repository root. But the moment we start setting inheritable properties on subtrees, we'd also need to be sure that those subtrees have the special permission if all users with access under that subtree don't have access to the root of the subtree. To condense to its essential core what I am proposing, it's this simple rule: If a user has read access to a path, then that path can inherit inheritable properties from its path-wise ancestors, regardless of the user's permissions to those ancestors And lastly, we haven't yet hammered out the issues surrounding what to do with potentially conflicting properties within the tree--though that might have been specific to log message templates, now that I think about it. We could chose to make things
Re: [RFC] Server Dictated Configuration
On Tue, Jan 17, 2012 at 5:36 PM, Paul Burba ptbu...@gmail.com wrote: On Mon, Jan 16, 2012 at 8:28 PM, Hyrum K Wright hyrum.wri...@wandisco.com wrote: [...] Another thing to note is that there have been some rumblings about authz improvements, along the lines of an additional permission to say you can know about this directory. I know C-Mike has been thinking about this off-and-on since the 3242 debacle, and something like inheritable props my fit in that model, though I'd had to make the feature dependency tree another level deeper. Even if that was implemented today it's still an administrative nightmare, albeit a lesser one. In the example above we'd need to give the new special permission to the repository root. But the moment we start setting inheritable properties on subtrees, we'd also need to be sure that those subtrees have the special permission if all users with access under that subtree don't have access to the root of the subtree. But, but ... if you're able to checkout ^/foo/bar/baz, then you already know that foo and foo/bar exist, don't you? If nothing else, 'svn info' will tell you that. So essentially, if we're talking about path-wise ancestors, you always know about those ancestors, there's no need to specifically configure this in any way. To condense to its essential core what I am proposing, it's this simple rule: If a user has read access to a path, then that path can inherit inheritable properties from its path-wise ancestors, regardless of the user's permissions to those ancestors Yep, makes perfect sense (exactly like you know about the existence of those paths already, by being able to checkout them). -- Johan
Re: [RFC] Server Dictated Configuration
On Tue, Jan 17, 2012 at 8:55 PM, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 8:33 AM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 12:51 AM, Paul Burba ptbu...@gmail.com wrote: [...] We could chose to make things simple and follow the svn:mergeinfo model of inheritance: a) If a path has an explicit given property then it doesn't inherit that property; the explicit value is the complete value. b) If a path doesn't explicitly have a given property then it inherits that property from its nearest parent with the property explicitly set on it. No merging of inherited with explicit properties, so no conflicts, just keep it simple. I'm just thinking out loud here, but what about the following approach: - These new svn properties (e.g. all properties in de 'svn:conf:' namespace) are always *inheritable*. - But whether or not they are *inherited* (and in what way) is up to the sub-tree. 'In what way': I'm thinking about inheritance vs. extending/appending vs. override. Hi Johan, (The following may be the same as what you describe above, but just to be clear...) Let's keep in mind we (or at least I :-) have been talking about two closely related, but ultimately separate ideas: 1) Generic inherited properties 2) How to use inherited properties to facilitate repository-dictated auto-props and global-ignores. Agreed. Let's take repository dictated auto-props and global-ignores as an example of inherited properties in general. Viewed from the perspective of #1, a subtree either: a) Explicitly has a property set on it b) Doesn't have that explicit property but inherits it from a path-wise ancestor c) Doesn't have the the explicit property nor does it inherit it from any ancestor. Ok. But we can also dispense with c, as long as we're talking about inheritable props: we could say that they are always inherited, no matter what (except if you override them of course). Which makes me wonder: how will we discern inheritable props from normal ones? Merely saying that the svn:conf: namespace (for instance) means that it's always inheritable, will not cut it if we're talking about a generic feature ... Those are the only three options. A subtree in the repository doesn't get to say whether or not in inherits a given property or not, either it does or does not by definition. Now as to what to *do* with this particular property... ...Once we start talking about #2 (or any other particular instance of an inheritable property), then we can imagine that different semantics come into play depending on the particular property and the context in which it is used. A subtree may inherit a given property, but what the client code decides to do with this value (ignore it, use it as is, merge it with something else, etc.) depends on context. I'm thinking a bit along the lines of OO-languages: some special syntax or keyword in the subtree's property signals that the property must be inherited. Let's say we use '$super' to point to the inherited prop, and if the prop is not specified, '=$super' is implied, then we could do: ^/ - svn:conf:ignore = *.o *.lo *.la *.al .libs ^/native-app - no prop needed ($super is implied) # inherit ^/native-app/python-mods - svn:conf:ignore = $super *.pyc # append ^/java-lib - svn:conf:ignore = *.class # override ^/java-lib/python-mods - svn:conf:ignore = $super *.pyc # append Or something like that. Hmmm, maybe, but as I've mentioned elsewhere, it's these types of complexities that have kept inheritable properties from being implemented before now. I can readily vouch for the fact that the non-inheritable vs. inheritable concept used with svn:mergeinfo, while it works, is difficult for users to understand and has been the cause of many an edge case bug. So as far as *generic* inheritable properties go I still prefer the override-only model. I'm not sure if my suggestion implies the same kind of complexities as with svn:mergeinfo. But then again, my involvement with subversion pre-dates merge tracking, so I'm not aware of all the discussions back then, ideas that were tossed around, ... and I certainly don't have the same battle-scars and experience with those things as (some of) you have :-). But anyway, starting out with an override-only system is perfectly fine. If it fixes repository-dictated auto-props, it will make a lot of svn admins (and users) very happy :-). It's still possible to add some form of extendability later if and when it's deemed useful. Thanks for your work on this. -- Johan
Re: [RFC] Server Dictated Configuration
On Tue, Jan 17, 2012 at 9:12 PM, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 17, 2012 at 2:54 PM, Johan Corveleyn jcor...@gmail.com wrote: On Tue, Jan 17, 2012 at 5:36 PM, Paul Burba ptbu...@gmail.com wrote: On Mon, Jan 16, 2012 at 8:28 PM, Hyrum K Wright hyrum.wri...@wandisco.com wrote: [...] Another thing to note is that there have been some rumblings about authz improvements, along the lines of an additional permission to say you can know about this directory. I know C-Mike has been thinking about this off-and-on since the 3242 debacle, and something like inheritable props my fit in that model, though I'd had to make the feature dependency tree another level deeper. Even if that was implemented today it's still an administrative nightmare, albeit a lesser one. In the example above we'd need to give the new special permission to the repository root. But the moment we start setting inheritable properties on subtrees, we'd also need to be sure that those subtrees have the special permission if all users with access under that subtree don't have access to the root of the subtree. But, but ... if you're able to checkout ^/foo/bar/baz, then you already know that foo and foo/bar exist, don't you? If nothing else, 'svn info' will tell you that. So essentially, if we're talking about path-wise ancestors, you always know about those ancestors, there's no need to specifically configure this in any way. Sure, *if* a path we have read access to can inherit an inheritable property from a parent we have no access to, then there is no problem. I was talking about the case where we want inheritable properties but don't want to allow inheritance in the case where we have no access to the parent. I was trying to make the argument that these authz improvements alone (at least as I understand them, which isn't very well) are still problematic in this case. Does that make sense? Yep, perfectly. Agreed that, whatever way it's implemented, you should be able to read the inheritable props from ancestor paths, even if you don't have authz access to those paths themselves. To condense to its essential core what I am proposing, it's this simple rule: If a user has read access to a path, then that path can inherit inheritable properties from its path-wise ancestors, regardless of the user's permissions to those ancestors Yep, makes perfect sense (exactly like you know about the existence of those paths already, by being able to checkout them). Not to argue against myself, but I am proposing that we'll know more than mere existence, we'll know the a particular property is set and that property's value. That information may come from a path we do not otherwise have access to. Some folks might object to this (Hyrum certainly was wary of it). My response is: If you purposefully set an *inheritable* property on a parent path and then gave access to a child path, don't be surprised if the child can inherit that property. Yep, agreed. -- Johan
Re: [RFC] Server Dictated Configuration
Hi All, I haven't been ignoring this thread, I've been laid low with illness the last couple of weeks... ...Anyhow, when I finally got back to this I see that a lot of folks are itching for an inheritable property solution. So for now I'll defer on the open questions re the proposal in the wiki. Instead let's talk inherited properties. I suspect that if we don't view inheritable props through an everything an pony too lens, if we agree on some basic limitations and easy to explain, well-defined behavior, then we can make this work[1]. [1] Understand that I am emotionally scarred from implementing mergetracking/mergeinfo, where we tried to include the pony. We got the pony in, but it trampled me on its way -- I don't want to go down that road again! On Thu, Jan 5, 2012 at 4:17 PM, Konstantin Kolinko knst.koli...@gmail.com wrote: I like TortoiseSVN way because a) it is already known for several years and it works b) it leverages existing support for path-based security, for svnsync and dump formats. On Thu, Jan 5, 2012 at 2:42 PM, Ivan Zhakov i...@visualsvn.com wrote: I think most of use-cases can be solved by existing mechanism without inventing something new: 1. auto-props TortoiseSVN already has 'tsvn:auto-props' property [1]. Which used to automatically set properties for added files. It would be nice if Subversion core support this property. I don't think we want to leverage tsvn:autoprops because the current Tortoise behavior is lacking some functionality we almost certainly want. For example, while TSVN can inherit the tsvn:autoprops property from within a working copy it can't inherit from the repository if the WC doesn't have the required property set. Also tsvn:autoprops affects imported files only when those files are imported into a directory with tsvn:autoprops explicitly set (there is no repository-side inheritance akin to what happens for additions in the WC). Here's a use-case keep in mind that shows what I think we need to support at a minimum: A repository like svn.apache.org/repos/asf which has many (over 100 in this case) projects that, while they have some common organizational requirements, are largely independent and quite likely have differing requirements for autoprops and ignores. A project administrator should be able to set an inheritable svn:auto-props property on the root of their project (i.e. s.a.o/repos/asf/subversion) and have it apply to across the project via inheritance. So even if a user checks out s.a.o/repos/asf/subversion/trunk then adds new files to their WC the auto-props should apply. If we can't support a basic use-case like this I think we need to pursue an alternative solution that does (like that in the wiki). So if we are going to do this right, we need to implement true inheritable properties, and use our own property (e.g. svn:auto-props) rather than trying to support tsvn:autoprops. I don't believe we want different behaviors for tsvn:autoprops when using the command line vs. Tortoise. 2. ignores We can add svn:global-ignores property to define global (recursive) ignore mask. Which requires inheritable properties right? 3. store-plaintext-passwords Since most used of platforms already supports password encryption (Windows, MacOS, KDE, GNOME) I think we can safely just change to do not store plaintext by default. As you may have seen, Mike Pilato is investigating the problem of credential caching, see http://wiki.apache.org/subversion/EncryptedPasswordStorage So I'm limiting my responses to inheritable properties in general and auto-props and global-ignores in particular. On Thu, Jan 5, 2012 at 3:03 PM, Mark Phippard markp...@gmail.com wrote: The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. Exactly, so what would an inheritable solution look like in this case? (Yes, lot of this is similar to what Konstantin suggested earlier in this thread). Say our magic inherited svn:auto-props property is set on ^/subversion. For the sake of completeness let's first look at the simple case: We check out ^/subversion. Here things are relatively easy: 1) When required we walk the WC and find the necessary inheritable property on the root of the WC. Done. Even here there complications to consider. How to handle mixed-rev working copies being the simplest: With svn:mergeinfo we don't inherit across mixed-rev boundries, because svn:mergeinfo is in a constant state of flux and we can easily contact the repository to obtain the correct inherited mergeinfo (a merge by definition needs contact with the repos, so there are no worries about disconnected
Re: [RFC] Server Dictated Configuration
On Fri, Jan 6, 2012 at 8:27 AM, Ivan Zhakov i...@visualsvn.com wrote: On Fri, Jan 6, 2012 at 00:03, Mark Phippard markp...@gmail.com wrote: On Thu, Jan 5, 2012 at 2:42 PM, Ivan Zhakov i...@visualsvn.com wrote: On Wed, Jan 4, 2012 at 02:58, Paul Burba ptbu...@gmail.com wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. I think most of use-cases can be solved by existing mechanism without inventing something new: 1. auto-props TortoiseSVN already has 'tsvn:auto-props' property [1]. Which used to automatically set properties for added files. It would be nice if Subversion core support this property. 2. ignores We can add svn:global-ignores property to define global (recursive) ignore mask. The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. That is why I believe it makes sense for SVN to support it natively using an approach something like described in the wiki. Or at least weigh that approach versus using properties within the repository. Perhaps properties are the way to add the deferred feature of supporting overrides based on path? Well, inherited properties would be nice. And may be we should spend our efforts to implement them. But even without inherited properties TSVN solution solves many real world use-cases. Come to think of it: using properties has the nice advantage that these configurations come along when a project is branched or tagged. Which is what most users would expect, I guess. If using some other server-side configuration mechanism, we'll surely have to provide some wildcard support to be able to set a configuration on a certain subtree *and all its branches and tags* (and even then, this isn't the same as taking the configuration with you to a new branch). -- Johan
Re: [RFC] Server Dictated Configuration
On 01/06/2012 07:16 AM, Johan Corveleyn wrote: Come to think of it: using properties has the nice advantage that these configurations come along when a project is branched or tagged. Which is what most users would expect, I guess. That's a great point, and echoes a problem which currently plagues our path-based authz stuff today. -- C. Michael Pilato cmpil...@collab.net CollabNet www.collab.net Distributed Development On Demand signature.asc Description: OpenPGP digital signature
Re: [RFC] Server Dictated Configuration
Johan Corveleyn wrote on Fri, Jan 06, 2012 at 13:16:06 +0100: Come to think of it: using properties has the nice advantage that these configurations come along when a project is branched or tagged. Which is what most users would expect, I guess. If using some other server-side configuration mechanism, we'll surely have to provide some wildcard support to be able to set a configuration on a certain subtree *and all its branches and tags* (and even then, this isn't the same as taking the configuration with you to a new branch). Another option would be to define a configuration for, say, the /trunk fspath, and when checking out a branch trace back through copies/renames to find this. This requires the 'branch root' concept (which Julian started on at some point). The configuration would then be maintained for /subversion/trunk and all branches would automagically inherit it too. It could either physically walk history (svn_ra_get_location_segments()) or shortcut and use the created path (FSFS cpath node-rev attribute) as the index to look up. Daniel (and yes, the latter would break up if /trunk@50 and /trunk@100 are not related...)
Re: [RFC] Server Dictated Configuration
On Thu, Jan 5, 2012 at 1:56 AM, Daniel Shahaf danie...@elego.de wrote: Johan Corveleyn wrote on Thu, Jan 05, 2012 at 01:40:40 +0100: Another question: is the server-dictated config still extendable by the client? I.e.: if the server already defines a global svn:ignore value, can the user still append another pattern? Or for autoprops: maybe I want to have my own extra autoprops in addition to the ones that are standardized by my organization ... I think that would be a quite desirable feature, though I'm uncertain about the details (should the client be able to decide whether to override or extend? how? With some special syntax in the config values? ...). Just wondering ... What if the server specifies *.c = svn:eol-style=native and the user's config specifies *.c = svn:keywords=HeadURL ? Do both of them take effect? (This may be what you asked.) Indeed, that's my question. Or: Server specifies global-ignores = *.o ... User happens to spread *.class files around in his working copy and wants to add: global-ignores = *.class But, if at some point we might want to allow (well-behaved) clients to override certain properties (if so allowed by the server), we might need to make the distinction between overriding and extending. Maybe some special syntax would help, to indicate extension? *.c += svn:keywords=HeadURL or *.c = ;svn:keywords=HeadURL or something like that. Oh, but what about multi-valued auto-props: Server specifies: *.c = svn:keywords=Author Date User wants to also add HeadURL as keyword: *.c += svn:keywords=HeadURL Hm, that's really an edge case, probably not that interesting :-) ... (and the problem is somewhat similar with the various ways of specifying properties that exist today (command-line options, user config file, system config file, ...)) Maybe overriding would also need a special syntax in the future, to avoid back-compat problems with old config files still lying around: *.c == svn:keywords=HeadURL or something like that ... -- Johan
Re: [RFC] Server Dictated Configuration
On Thu, Jan 5, 2012 at 2:42 PM, Ivan Zhakov i...@visualsvn.com wrote: On Wed, Jan 4, 2012 at 02:58, Paul Burba ptbu...@gmail.com wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. I think most of use-cases can be solved by existing mechanism without inventing something new: 1. auto-props TortoiseSVN already has 'tsvn:auto-props' property [1]. Which used to automatically set properties for added files. It would be nice if Subversion core support this property. 2. ignores We can add svn:global-ignores property to define global (recursive) ignore mask. The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. That is why I believe it makes sense for SVN to support it natively using an approach something like described in the wiki. Or at least weigh that approach versus using properties within the repository. Perhaps properties are the way to add the deferred feature of supporting overrides based on path? 3. store-plaintext-passwords Since most used of platforms already supports password encryption (Windows, MacOS, KDE, GNOME) I think we can safely just change to do not store plaintext by default. You must not spend time working with Unix users. The options we provided are good, but do not seem to be enough and there is no way to enforce policies where they have to be used. It also does nothing to help things like automated build processes and scripts that need to run on Unix and cannot interact with a keyring. I do not pretend to have an answer though. Making plain text off by default might be a start. I think some people might object to adding security theater by using something like AES encryption to obfuscate plain text passwords but I think some of our users would appreciate that option over plain text. It seems like we could possibly add an option for the user to provide some salt when configuring and building the binaries as well. I am guessing this could be read from the strings in the compiled library but it would still provide a small measure of additional obfuscation that someone would need to know. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: [RFC] Server Dictated Configuration
Ivan Zhakov wrote on Thu, Jan 05, 2012 at 23:42:49 +0400: 2. ignores We can add svn:global-ignores property to define global (recursive) ignore mask. Glossing over details. Is it a revprop? What happens if the wc root is not the branch root?
Re: [RFC] Server Dictated Configuration
On Thu, Jan 5, 2012 at 3:23 PM, Stefan Küng tortoise...@gmail.com wrote: The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. That is why I believe it makes sense for SVN to support it natively using an approach something like described in the wiki. Or at least weigh that approach versus using properties within the repository. Perhaps properties are the way to add the deferred feature of supporting overrides based on path? overrides based on path - do you mean that properties would be inherited, i.e., child items inherit the properties of the parent item(s)? Yes. Presumably. Imagine the ASF repository with hundreds of autonomous projects. Ideally each project could have its own settings but still live in the same repository. And the best advantage of all: users could set/modify the properties with their clients. No need to contact the administrator and have him do it - believe me, Admins will very soon get annoyed if such changes/modifications happen often. That is good for some settings, but not necessarily all of them, although I suppose an Admin could use a pre-commit hook script to prevent clients from committing settings changes if they are not desired. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: [RFC] Server Dictated Configuration
On 03.01.2012 23:58, Paul Burba wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. Looks good. Just a few thoughts: * are there namespaces planned (like 'svn:' for the properties) so other clients can use this feature as well for their own settings? * is only an admin able to change these configs or can users with commit access do it as well? If not, I think this should be considered as well since involving an admin in big companies is always a real pain... Stefan -- ___ oo // \\ De Chelonian Mobile (_,\/ \_/ \ TortoiseSVN \ \_/_\_/The coolest Interface to (Sub)Version Control /_/ \_\ http://tortoisesvn.net
Re: [RFC] Server Dictated Configuration
2012/1/6 Mark Phippard markp...@gmail.com: On Thu, Jan 5, 2012 at 2:42 PM, Ivan Zhakov i...@visualsvn.com wrote: http://wiki.apache.org/subversion/ServerDictatedConfiguration I think most of use-cases can be solved by existing mechanism without inventing something new: 1. auto-props TortoiseSVN already has 'tsvn:auto-props' property [1]. Which used to automatically set properties for added files. It would be nice if Subversion core support this property. 2. ignores We can add svn:global-ignores property to define global (recursive) ignore mask. The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. As I wrote in this very thread two days ago [1], this shortcoming could be solved if the properties for the parent folders were present in the WC. Now that wc has a database it could store those properties. [1] http://markmail.org/message/np5bfcomaci5u3c5 That is why I believe it makes sense for SVN to support it natively using an approach something like described in the wiki. Or at least weigh that approach versus using properties within the repository. Perhaps properties are the way to add the deferred feature of supporting overrides based on path? I like TortoiseSVN way because a) it is already known for several years and it works b) it leverages existing support for path-based security, for svnsync and dump formats. If this approach is rejected one could at least mention that it was considered and write down why it was rejected. Best regards, Konstantin Kolinko
Re: [RFC] Server Dictated Configuration
On Thu, Jan 5, 2012 at 4:17 PM, Konstantin Kolinko knst.koli...@gmail.com wrote: The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. As I wrote in this very thread two days ago [1], this shortcoming could be solved if the properties for the parent folders were present in the WC. Now that wc has a database it could store those properties. I think the last time this feature (server-dictated configuration) was proposed, this was the approach considered. Perhaps the new WC format makes it more possible than last time, I do not know. It seems like it would be less efficient though. The approach outlined in the proposal, via its usage of the hash and comparing the hash when the session is opened makes it easy for the client to know when settings have changed. If we use properties in the manner described, then doesn't every RA session have to inspect every folder all the way back to the root on every request? It seems like someone would have to come up with a way to solve this problem, or remove the requirement of inheriting properties that are not already in the WC. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: [RFC] Server Dictated Configuration
On Thu, Jan 5, 2012 at 3:23 PM, Mark Phippard markp...@gmail.com wrote: On Thu, Jan 5, 2012 at 4:17 PM, Konstantin Kolinko knst.koli...@gmail.com wrote: The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. As I wrote in this very thread two days ago [1], this shortcoming could be solved if the properties for the parent folders were present in the WC. Now that wc has a database it could store those properties. I think the last time this feature (server-dictated configuration) was proposed, this was the approach considered. Perhaps the new WC format makes it more possible than last time, I do not know. It seems like it would be less efficient though. The approach outlined in the proposal, via its usage of the hash and comparing the hash when the session is opened makes it easy for the client to know when settings have changed. If we use properties in the manner described, then doesn't every RA session have to inspect every folder all the way back to the root on every request? It seems like someone would have to come up with a way to solve this problem, or remove the requirement of inheriting properties that are not already in the WC. We've been talking about a general mechanism for inherited properties for a *long* time. If implemented, such a mechanism could be used for repos-dictated config through inherited, versioned properties. Another possible use could be log message templates. As I recall, there were a few reasons why inherited properties haven't been implemented. One is the client-side storage and lookup, which wc-ng has helped with. The other is what to do with non-checked out parent directories, which you mention above. Another problem is the various authz issues, similar to the infamous issue 3242 problems we had with copy and move. And lastly, we haven't yet hammered out the issues surrounding what to do with potentially conflicting properties within the tree--though that might have been specific to log message templates, now that I think about it. In any case, there could be many uses for inherited properties, but not all the kinks have been worked out. Though it might be time to revisit them. -Hyrum -- uberSVN: Apache Subversion Made Easy http://www.uberSVN.com/
Re: [RFC] Server Dictated Configuration
On Fri, Jan 6, 2012 at 00:03, Mark Phippard markp...@gmail.com wrote: On Thu, Jan 5, 2012 at 2:42 PM, Ivan Zhakov i...@visualsvn.com wrote: On Wed, Jan 4, 2012 at 02:58, Paul Burba ptbu...@gmail.com wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. I think most of use-cases can be solved by existing mechanism without inventing something new: 1. auto-props TortoiseSVN already has 'tsvn:auto-props' property [1]. Which used to automatically set properties for added files. It would be nice if Subversion core support this property. 2. ignores We can add svn:global-ignores property to define global (recursive) ignore mask. The approach TortoiseSVN and some other clients take does work pretty nicely but I also think they reveal the short comings in using properties. For convenience, TortoiseSVN does not force you to set these properties on every folder and instead will walk to the root of your WC to find them, but then this also exposes the problem that if you did not checkout the folder that has those properties you are back to square one. That is why I believe it makes sense for SVN to support it natively using an approach something like described in the wiki. Or at least weigh that approach versus using properties within the repository. Perhaps properties are the way to add the deferred feature of supporting overrides based on path? Well, inherited properties would be nice. And may be we should spend our efforts to implement them. But even without inherited properties TSVN solution solves many real world use-cases. 3. store-plaintext-passwords Since most used of platforms already supports password encryption (Windows, MacOS, KDE, GNOME) I think we can safely just change to do not store plaintext by default. You must not spend time working with Unix users. That's true: I live in wonderful Windows where Group Policy could serve everything :) The options we provided are good, but do not seem to be enough and there is no way to enforce policies where they have to be used. It also does nothing to help things like automated build processes and scripts that need to run on Unix and cannot interact with a keyring. I do not pretend to have an answer though. Making plain text off by default might be a start. I don't pretend that my proposal solves all problems, but I think they should notably improve situation for users. -- Ivan Zhakov
Re: [RFC] Server Dictated Configuration
On Fri, Jan 6, 2012 at 00:10, Daniel Shahaf d...@daniel.shahaf.name wrote: Ivan Zhakov wrote on Thu, Jan 05, 2012 at 23:42:49 +0400: 2. ignores We can add svn:global-ignores property to define global (recursive) ignore mask. Glossing over details. Is it a revprop? What happens if the wc root is not the branch root? Regular property. It won't work if you checkout part of branch root. We need inherited properties to solve this use case. -- Ivan Zhakov
Re: [RFC] Server Dictated Configuration
Paul Burba wrote on Wed, Jan 04, 2012 at 13:30:41 -0500: On Tue, Jan 3, 2012 at 6:16 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: On Tue, Jan 3, 2012, at 17:58, Paul Burba wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. 2. How would failure to create ~/.subversion/repos/${UUID}/foo be considered? Fatal error? Warning-but-continue? I'd say the latter, which is what we do now if Subversion cannot create the standard run-time config: +1, for reasons similar to yours. My reasoning for this is that we already accept that fact that a malicious user can hack up a custom client to ignore the server dictated config. We still expect the repository administrator to enforce (where possible) their desired configuration via hook scripts. All we are trying to do here is make it possible for well-behaved clients to *easily* do the right thing. 3. Re password storage: does it make sense to allow the server to push a may store plaintext passwords setting to the client? I'm with you to the extent that that doesn't make much sense... That is a security risk, for example, in environments where the password is not transmitted on the wire. ...but I don't follow how exactly this is a security risk. Could you elaborate? Sure: I think it's a greater security risk for the server to cause the client to record on disk a password that the server admin doesn't know, than the same with a password the server admin knows. The generic case here is Allow option X to be pushed only to some of its possible values. 4. In Related issues, add the title or description of each issue alongside each number? (Thanks.) Done. Thanks. 5. wrt the '${ASF_REPOS_UUID}:/subversion' use-case, how forward- compatible is the proposed design to extensions such as that one? (Not saying that specific extension needs to be possible, but we'll probably want to extend this feature in various ways in 1.9 and we don't want it to be as hard as FSFS to extend.) By the '${ASF_REPOS_UUID}:/subversion' use-case you mean deferred goal #4? [[[ ... hierarchical configuration, users could configure such things as in all working copies of ${ASF_REPOS_UUID}:/subversion, do not store pristines. ]]] Yes. IIUC you want to know how easily the proposed server dictated config client-side storage model, i.e. a UUID-keyed subdir for every repository supplied configuration, would work with an extended client-side configuration model. Yes. I'm assuming that once we release this feature in 1.8, we will want to extend it in 1.9. I don't know yet _how_ we'll want to extend it (deferred goal #4 is but one route we might go down), but I'm asking how amenable to extensibility is the new format. (Concrete example: what happens when a 1.9 client writes a ~/.s/repos/ entry, and a 1.8 client then tries to read it?) What is proposed for server dictated config is obviously not a perfect fit for client side config, e.g. do we still have a global ${HOME}/.subversion/config file, and if so, how does a UUID-keyed local config override it...but I don't see any deal-breakers here for the current proposal. OK. Paul Thanks, Daniel
Re: [RFC] Server Dictated Configuration
On Tue, Jan 3, 2012 at 6:33 PM, Stefan Sperling s...@elego.de wrote: On Tue, Jan 03, 2012 at 05:58:16PM -0500, Paul Burba wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. Paul I did read it today and I like it. One open question for me was how do we treat invalid configuration options in the server-side configuration file? This is one of the possible failure modes the design should address. I see several options here: 1) The simple option: Allow admins to write a syntactically invalid config on the server. If they neglect to test it, then the invalid config gets sent to the client. Of course the client can't parse it, but all that happens is that the client issues a warning and then allows the operation to succeed. Why? I fall back on the same logic I used in response to Daniel's question about what to do if we can't create a config cache in the first place: ...we already accept that fact that a malicious user can hack up a custom client to ignore the server dictated config. We still expect the repository administrator to enforce (where possible) their desired configuration via hook scripts. All we are trying to do here is make it possible for well-behaved clients to *easily* do the right thing. This isn't very different from an repository admin creating a bad authz that grants unintended access. 2) As per #1 but we also add a new svnadmin subcommand ('confcheck'?) that parses the server side config and checks that only valid server-side config options are set. This is simple and but can obviously be skipped (then we fall back to #1?). Maybe we want to make this test everything in ${REPOS_PATH}/conf/*.* by default when given a repository path and text a particular ini file if given a file path (which would be a nice-to-have for Apache authz files which don't live in the repository tree. 3) A new svnadmin subcommand ('confset'?) that installs a configuration file, parsing and testing it, and if everything is ok, moving it into ${REPOS_PATH}/conf/repos.conf. This would ensure that any changes to the server-side config are well-formed. But it is ugly, because it breaks with our tradition of simply being to hand-edit authz and config files in-place. Also I'm obviously doing a lot of hand-waving regarding how we prevent/detect/recover from such hand-edits! 4) Anytime a client requests a copy of the configuration from the server, the server parses and tests the latest configuration. If any errors are found then the server returns and error describing the problem. This is harsh, in that it makes the repository essentially unusable, but an admin who goes through the trouble of writing a server dictated config probably wants to know the config is bogus and doesn't mind(?) locking the repos until such time as it is fixed. #2 and #4 can obviously coexist Say there is a syntax error in the configuration file which prevents us from parsing it. Do we alert the administrator? If so, how? Or will it fail silently? I hadn't considered direct notification of the repos admin by the client. Only #3 immediately confronts the repository admin with the fact they have a bogus config. The other options require a purposeful step on the part of the admin or for a user to complain (and in the case of #1 this might not happen since the user can still do what they want). If the option name is valid but the option value is not, will the server still send the configuration option to the client? Will it send some default value instead of the invalid value? Or will it omit the option from the list of configuration options sent to the client? I'll defer on this until we come to some consensus on the above (but with options #1 and #2 this is certainly an open question). Paul
Re: [RFC] Server Dictated Configuration
On Wed, Jan 4, 2012 at 7:30 PM, Paul Burba ptbu...@gmail.com wrote: On Tue, Jan 3, 2012 at 6:16 PM, Daniel Shahaf d...@daniel.shahaf.name wrote: On Tue, Jan 3, 2012, at 17:58, Paul Burba wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. 1. Under So, the order in which specific configuration options will be honored where found is:, you say that ~/.subversion/ settings would be overriden by /etc/subversion settings. That's not how the code works today --- i.e., not a compatible change. Typo? I intended that list to be from highest priority to lowest, just clarified that. Also added command-line options to the list at the number 2 spot. I think command-line options should still have number 1 priority, so overrule the server-side suggestions. People use command-line options usually very consciously, when they're trying to do something special that's outside of the normal usage. So I would expect command-line options to take priority. If things need to be enforced, they need to be checked on the server-side anyway, as already described on the wiki page. If the command-line options don't take priority, I can see workarounds appearing based on the following: 2. How would failure to create ~/.subversion/repos/${UUID}/foo be considered? Fatal error? Warning-but-continue? I'd say the latter, which is what we do now if Subversion cannot create the standard run-time config: Aha, so if I want my script, which tries to do special things with command-line options, to overrule the server-side config, let's move ~/.subversion/repos/${UUID}/foo temporarily away, and make the directory read-only. Another question: is the server-dictated config still extendable by the client? I.e.: if the server already defines a global svn:ignore value, can the user still append another pattern? Or for autoprops: maybe I want to have my own extra autoprops in addition to the ones that are standardized by my organization ... I think that would be a quite desirable feature, though I'm uncertain about the details (should the client be able to decide whether to override or extend? how? With some special syntax in the config values? ...). Just wondering ... -- Johan
Re: [RFC] Server Dictated Configuration
Johan Corveleyn wrote on Thu, Jan 05, 2012 at 01:40:40 +0100: Another question: is the server-dictated config still extendable by the client? I.e.: if the server already defines a global svn:ignore value, can the user still append another pattern? Or for autoprops: maybe I want to have my own extra autoprops in addition to the ones that are standardized by my organization ... I think that would be a quite desirable feature, though I'm uncertain about the details (should the client be able to decide whether to override or extend? how? With some special syntax in the config values? ...). Just wondering ... What if the server specifies *.c = svn:eol-style=native and the user's config specifies *.c = svn:keywords=HeadURL ? Do both of them take effect? (This may be what you asked.)
Re: [RFC] Server Dictated Configuration
Paul Burba wrote on Wed, Jan 04, 2012 at 14:44:02 -0500: On Tue, Jan 3, 2012 at 6:33 PM, Stefan Sperling s...@elego.de wrote: If the option name is valid but the option value is not, will the server still send the configuration option to the client? Will it send some default value instead of the invalid value? Or will it omit the option from the list of configuration options sent to the client? I'll defer on this until we come to some consensus on the above (but with options #1 and #2 this is certainly an open question). Valid is a relative notion. What if the server is 1.8, the client is 1.9, and the broadcast config includes a new-in-1.9 setting? Paul
Re: [RFC] Server Dictated Configuration
On Tue, Jan 3, 2012, at 17:58, Paul Burba wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. 1. Under So, the order in which specific configuration options will be honored where found is:, you say that ~/.subversion/ settings would be overriden by /etc/subversion settings. That's not how the code works today --- i.e., not a compatible change. Typo? 2. How would failure to create ~/.subversion/repos/${UUID}/foo be considered? Fatal error? Warning-but-continue? 3. Re password storage: does it make sense to allow the server to push a may store plaintext passwords setting to the client? That is a security risk, for example, in environments where the password is not transmitted on the wire. The generic case here is Allow option X to be pushed only to some of its possible values. 4. In Related issues, add the title or description of each issue alongside each number? (Thanks.) 5. wrt the '${ASF_REPOS_UUID}:/subversion' use-case, how forward- compatible is the proposed design to extensions such as that one? (Not saying that specific extension needs to be possible, but we'll probably want to extend this feature in various ways in 1.9 and we don't want it to be as hard as FSFS to extend.) Paul
Re: [RFC] Server Dictated Configuration
On Tue, Jan 03, 2012 at 05:58:16PM -0500, Paul Burba wrote: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. Paul I did read it today and I like it. One open question for me was how do we treat invalid configuration options in the server-side configuration file? This is one of the possible failure modes the design should address. Say there is a syntax error in the configuration file which prevents us from parsing it. Do we alert the administrator? If so, how? Or will it fail silently? If the option name is valid but the option value is not, will the server still send the configuration option to the client? Will it send some default value instead of the invalid value? Or will it omit the option from the list of configuration options sent to the client?
Re: [RFC] Server Dictated Configuration
2012/1/4 Paul Burba ptbu...@gmail.com: Mike Pilato and I have been kicking around some ideas on server dictated configuration recently and have put our thoughts into a wiki (full disclosure: this wiki was initially based on Hyrum's thoughts on the subject in https://svn.apache.org/repos/asf/subversion/trunk/notes/repos-dictated-config) : http://wiki.apache.org/subversion/ServerDictatedConfiguration We're at a point where it's time to solicit some wider feedback, so please have a look at the wiki and follow-up here with any concerns, thoughts, suggestions, etc.. 1. I do not see path-based authorization being mentioned. The proposal says that The current plan is to allow configuration at the most granular level, per repository-path. It means that path-based authorization must be applied to it and it must hide the configuration for those parts of repository that are not readable by the current user. I do not see how it plays with proposed Server-client transmission mechanism that uses a checksum and configuration caching at the client. If admin gives an user additional rights, so that she now sees a part of repository that she did not see before, does she need to reread the configuration? How a client can know that? Will the checksum change? Should the checksum depend on the path that is accessed by the client? 2. Note that TortoiseSVN already implements certain server-dictated configuration options. Documentation: [1] http://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-dug-propertypage.html#tsvn-dug-propertypage-tsvn-props In short: 1) They are implemented as svn properties that are set on directories 2) Notable ones are: - tsvn:logminsize sets the minimum length of a log message for a commit - tsvn:autoprops overrides Subversion' autoprops setting - tsvn:projectlanguage sets language that is used when spellchecking the log message - bugtraq:* properties provide intergration with bug tracking software. I like this feature but a drawback in the current implementation is that the properties must be present in the user's WC. If I checkout a subfolder of the original project and the properties are not set on that folder then TortoiseSVN does not see them. I think that now with WC-NG this drawback can be cured by the following change: - Allow the WC to know the properties not only of its subtree, but also of its parent folders up to the repository root. Mechanism to send configuration updates to the client: The client should update the root folder of its WC. When the root folder of WC is updated, the changes in the properties of its parent directories could be sent as well. Mechanism to enforce configuration updates: I think that it could be similar to what happens when you try to commit property changes on a directory. If your directory is not up to date the commit is rejected and you are asked to update your WC. I think that to ask the server for the properties of those parent folder the client can use the same protocol that is used in sparse checkouts. That is if it were a sparse checkout starting from the repository root. The only difference is that the parent folders are not actually present on the user's hard drive. (The client should use this feature only on svn 1.5 and later servers, because the feature needs proper server-side support of sparse checkouts.). Best regards, Konstantin Kolinko