Here's my current lens for managing the /etc/conf.d/net file for gentoo; it uses the @syntax to map between baselayout 1 and 2.
The big thing is the difference in handling quoting; the old lens would allow you to switch between quoting styles within a single variable (since it was a real array) but now it's forced to one quoting style per-array. I can think of situations where this might bite you (including literal $'s) but I can't think of a good way to handle it. -Doug On 04/25/2012 06:09 PM, Raphaël Pinson wrote: > On Wed, Apr 25, 2012 at 10:09 PM, Doug Warner <[email protected]> wrote: >> Sorry for the long delay here; starting to come back around to this problem >> :) >> >> I really like your idea for supporting multiple syntaxes here and marking >> them >> with this type of "version" string (you could label each syntax for each >> matching version to make it easier to understand what is going on). > > > You need to leave one syntax without a "@syntax" node (probably the > most recent syntax). The reason for this is that otherwise, this > parameter would become necessary for the tree to be valid, and users > would have to specify which syntax they want to use. It's much easier > if users don't care about it and just get the newest syntax by > default, without having to even know that "@syntax" nodes exist. > > >> So, related to this; would it be possible to include an external lens to >> handle the parsing for the "old version" and write the new version inside the >> current file? I was thinking of just continuing to use the ShellVars lens to >> parse the line for the "old version" and tag it the appropriate syntax >> version >> and then write the new one as the default. > > > Sure, that's possible, but there are conditions. The example I gave > was quite simple (even simplistic). You have to avoid ambiguity in the > Augeas bidirectional transformations. That means: > > * No ambiguity on keys (same key mapping different syntaxes with > different lenses); > * No ambiguity on syntax (overlapping regexes for several syntaxes). > > > In the example I gave, I handled these two cases by: > > * Using a single label (in that case "key") declaration to avoid the > ambiguity on the keys, and handling the syntax cases inside this > single lens (declaring 3 lenses and using a union of them wouldn't > work, it would raise an ambiguity since the 3 keys would match the > same regex); > * Making sure syntaxes are non-overlapping (in this case, using > colons, commas or equal sign as separators ensure they cannot be > overlapping). > > The lens declaration could very well use more complex lenses, and even > lenses from other modules (such as Shellvars.lns), so long as it > follows these two rules. > > If you have a specific case, that could be interesting. > > Raphaël > > > >> On 02/11/2012 05:07 PM, Raphaël Pinson wrote: >>> After thinking about it some more, I think it could be useful to get >>> to a standard using subnodes. >>> >>> Here is an example, unrelated to the suggested problem: >>> >>> >>> module Migration = >>> >>> (* There are 3 possible syntaxes for a line >>> The last one doesn't have a "@syntax" subnode, it's the default - >>> newest - syntax *) >>> let line = [ key Rx.word . >>> ( (Sep.colon . store Rx.word . [ label "@syntax" . value "1" ]) >>> | (Sep.space . store Rx.word . [ label "@syntax" . value "2" ]) >>> | (Sep.equal . store Rx.word) ) >>> . Util.eol ] >>> >>> let lns = line* >>> >>> (* Now for some tests *) >>> let full = "foo:bar\nfoo=baz\nfoo baa\n" >>> >>> (* Get all kind of syntaxes *) >>> test lns get full = >>> { "foo" = "bar" >>> { "@syntax" = "1" } >>> } >>> { "foo" = "baz" } >>> { "foo" = "baa" >>> { "@syntax" = "2" } >>> } >>> >>> (* Migrate all to new syntax *) >>> test lns put full after >>> rm "//@syntax" = >>> "foo=bar\nfoo=baz\nfoo=baa\n" >>> >>> (* Force syntax 1 for last entry *) >>> test lns put full after >>> set "/foo[3]/@syntax" "1" = >>> "foo:bar\nfoo=baz\nfoo:baa\n" >>> >>> >>> >>> Could that be an interesting approach in general? >>> >>> Raphaël >>> >>> >>> >>> 2012/1/26 Raphaël Pinson <[email protected]>: >>>> Hi Doug, >>>> >>>> On Thu, Jan 26, 2012 at 4:07 PM, Doug Warner <[email protected]> wrote: >>>>> I'm working on a lens to work with Gentoo's new /etc/conf.d/net format >>>>> and I >>>>> realized I need to convert their old format to the new format. >>>>> >>>>> What's the best way to migrate from one format to another? I have my lens >>>>> mostly working for the new format now, but I realized I might need to >>>>> parse >>>>> both versions; but then I thought that it might not update when augeas >>>>> reads/writes the config file. >>>>> >>>>> I am using puppet to manage these files; so the values are typically set >>>>> each >>>>> run; I'm hoping that would cause augeas to write down the new format >>>>> rather >>>>> than leave it the old version. >>>>> >>>>> I guess the last option is to wipe out the old file when I upgrade and let >>>>> puppet/augeas write down the new file, but that seems painful as well. >>>>> >>>>> Any recommendations on what to do here? >>>>> >>>> >>>> >>>> That's a very interesting question. I see two ways of doing this: >>>> * write a single lens which will map both syntaxes to different nodes >>>> in the tree, and use the "move" command to migrate; >>>> * or write two lenses bound to two different files (file.old and >>>> file.new or something similar) and use the "move" command to migrate. >>>> >>>> >>>> Here is a (minimalistic) proof of concept using the first option: >>>> >>>> $ cat myconfig.aug >>>> module MyConfig = >>>> >>>> autoload xfm >>>> >>>> let old_record = >>>> let entry = [ Util.del_str "\"" >>>> . label "entry" >>>> . store /[0-9.\/]+/ >>>> . Util.del_str "\"" ] >>>> in [ label "old_record" . store Rx.word >>>> . Util.del_str "=(" >>>> . Build.opt_list entry Sep.space >>>> . Util.del_str ")" . Util.eol ] >>>> >>>> let new_record = >>>> let entry = [ label "entry" >>>> . store /[0-9.\/]+/ ] >>>> in [ label "new_record" . store Rx.word >>>> . Util.del_str "=\"" >>>> . Build.opt_list entry Util.eol >>>> . Util.del_str "\"" . Util.eol ] >>>> >>>> >>>> let lns = old_record | new_record >>>> >>>> let filter = incl "/file" >>>> >>>> let xfm = transform lns filter >>>> >>>> $ cat fakeroot/file >>>> config_eth0=("192.168.0.1/24" "192.168.0.2/24") >>>> >>>> $ augtool -I . -r fakeroot >>>> /files >>>> /files/file >>>> /files/file/old_record = "config_eth0" >>>> /files/file/old_record/entry[1] = "192.168.0.1/24" >>>> /files/file/old_record/entry[2] = "192.168.0.2/24" >>>> rpinson@rpinson:~/bas/augeas_migration$ augtool -I . -r fakeroot/ >>>> augtool> print /files/ >>>> /files >>>> /files/file >>>> /files/file/old_record = "config_eth0" >>>> /files/file/old_record/entry[1] = "192.168.0.1/24" >>>> /files/file/old_record/entry[2] = "192.168.0.2/24" >>>> augtool> mv /files/file/old_record /files/file/new_record >>>> augtool> print /files/ >>>> /files >>>> /files/file >>>> /files/file/new_record = "config_eth0" >>>> /files/file/new_record/entry[1] = "192.168.0.1/24" >>>> /files/file/new_record/entry[2] = "192.168.0.2/24" >>>> augtool> save >>>> Saved 1 file(s) >>>> >>>> $ cat fakeroot/file >>>> config_eth0="192.168.0.1/24 >>>> 192.168.0.2/24" >>>> >>>> >>>> Note that you can combine this with the -b flag to keep the old version at >>>> hand. >>>> >>>> >>>> Raphaël >> >>
(* Lens to match the crazy newline-split variables that Gentoo *)
(* uses in it's openrc-based /etc/conf.d/net file *)
module GentooConf_net=
autoload xfm
let nl = Util.del_str "\n"
let eol = Util.eol
let key_re = /[A-Za-z0-9_]+/
let eq = Util.del_str "="
let comment = Util.comment
let empty = Util.empty
let indent = Util.indent
let char = /[^;#() '"\t\n]|\\\\"/
let dquot = /"([^"\\\n]|\\\\.)*"/ (* " Emacs, relax *)
let sqword = /[^('\n]+/
let dqword = /[^("\n]+/
let uqword = /[^("'\n]+/
let semicol_eol = del /[ \t]*[;\n]/ "\n"
let comment_eol = Util.comment_eol
let comment_or_eol = comment_eol | semicol_eol
(* handle old style bash arrays in new syntax *)
let v1_list =
let v1_list_value = store /[^;#()'"\t\n]+/ in
[ label "quote" . store /['"]/ ] . [ label "value" . v1_list_value ] .
del /['"]/ "\"" .
[ Util.del_ws_spc . del /['"]/ "\"" . label "value" . v1_list_value . del
/['"]/ "\"" ]*
let v1_arr = del /\([ \t]*/ "(" . v1_list . del /[ \t]*\)/ ")"
let v1_simple = del /\(/ "(" . [ label "value" . empty ] . del /\)/ ")"
(* handle lists of arrays *)
let list(word:regexp) =
let list_value = store word in
[ label "value" . list_value ] .
[ nl . label "value" . list_value ]*
(* handle single quoted lists *)
let squote_arr = [ label "quote" . store /'/ ] . (list sqword)? . del /'/ "'"
(* similarly handle double qouted lists *)
let dquote_arr = [ label "quote" . store /"/ ] . (list dqword)? . del /"/ "\""
(* handle unquoted single value *)
let unquot_val = [ label "quote" . store "" ] . [label "value" . store
uqword+]?
(* the source/. inclusion method of bash *)
let source =
[ Util.indent
. del /\.|source/ "." . label ".source"
. Util.del_ws_spc . store /[^;=# \t\n]+/ . comment_or_eol ]
(* lens for key value pairs *)
let kv = [ key key_re . eq .
( ( v1_arr . [ label "@syntax" . value "1" ] )
| ( v1_simple . [ label "@syntax" . value "1" ] )
| squote_arr
| dquote_arr
| unquot_val
) . nl ]
| source
let lns = ( comment | empty | kv )*
let filter = incl "/etc/conf.d/net*"
. Util.stdexcl
let xfm = transform lns filter
(* Local Variables: *)
(* mode: caml *)
(* End: *)
module Test_GentooConf_Net=
let complicated_bridge_vlan_old_arrays = "config_eth0=(\"null\")
config_eth1=(\"null\")
vconfig_eth1=(\"set_name_type VLAN_PLUS_VID_NO_PAD\")
vlans_eth1=\"10 11 15\"
config_vlan10=(\"null\")
bridge_br10=(\"vlan10\")
routes_br10=(\"default via 10.128.0.254\")
config_br10=(\"10.128.1.11/22\")
RC_NEED_br10=\"net.eth1\"
brctl_br10=(\"setfd 0\")
bridge_br11=(\"vlan11\")
config_vlan11=(\"null\")
brctl_br11=(\"setfd 0\")
RC_NEED_br11=\"net.eth1\"
config_br11=(\"null\")
RC_NEED_br15=\"net.eth1\"
brctl_br15=(\"setfd 0\")
config_br15=(\"null\")
bridge_br15=(\"vlan15\")
config_vlan15=(\"null\")
vlan_start_eth1=\"no\"
RC_NEED_vlan11=\"net.eth1\"
RC_NEED_vlan10=\"net.eth1\"
RC_NEED_vlan15=\"net.eth1\"
"
let complicated_bridge_vlan = "config_eth0=\"null\"
config_eth1=\"null\"
vconfig_eth1=\"set_name_type VLAN_PLUS_VID_NO_PAD\"
vlans_eth1=\"10 11 12\"
config_vlan10=\"null\"
bridge_br10=\"vlan10\"
routes_br10=\"default via 192.168.0.254\"
config_br10=\"192.168.0.1/22\"
RC_NEED_br10=\"net.eth1\"
brctl_br10=\"setfd 0\"
bridge_br11=\"vlan11\"
config_vlan11=\"null\"
brctl_br11=\"setfd 0\"
RC_NEED_br11=\"net.eth1\"
config_br11=\"null\"
RC_NEED_br12=\"net.eth1\"
brctl_br12=\"setfd 0\"
config_br12=\"null\"
bridge_br12=\"vlan12\"
config_vlan12=\"null\"
vlan_start_eth1=\"no\"
RC_NEED_vlan11=\"net.eth1\"
RC_NEED_vlan10=\"net.eth1\"
RC_NEED_vlan12=\"net.eth1\"
"
(* FIXME need to parse the old bash arrays *)
test GentooConf_Net.lns get complicated_bridge_vlan_old_arrays =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
{ "config_eth1"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
{ "vconfig_eth1"
{ "quote" = "\"" }
{ "value" = "set_name_type VLAN_PLUS_VID_NO_PAD" }
{ "@syntax" = "1" }
}
{ "vlans_eth1"
{ "quote" = "\"" }
{ "value" = "10 11 15" }
}
{ "config_vlan10"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
{ "bridge_br10"
{ "quote" = "\"" }
{ "value" = "vlan10" }
{ "@syntax" = "1" }
}
{ "routes_br10"
{ "quote" = "\"" }
{ "value" = "default via 10.128.0.254" }
{ "@syntax" = "1" }
}
{ "config_br10"
{ "quote" = "\"" }
{ "value" = "10.128.1.11/22" }
{ "@syntax" = "1" }
}
{ "RC_NEED_br10"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "brctl_br10"
{ "quote" = "\"" }
{ "value" = "setfd 0" }
{ "@syntax" = "1" }
}
{ "bridge_br11"
{ "quote" = "\"" }
{ "value" = "vlan11" }
{ "@syntax" = "1" }
}
{ "config_vlan11"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
{ "brctl_br11"
{ "quote" = "\"" }
{ "value" = "setfd 0" }
{ "@syntax" = "1" }
}
{ "RC_NEED_br11"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "config_br11"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
{ "RC_NEED_br15"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "brctl_br15"
{ "quote" = "\"" }
{ "value" = "setfd 0" }
{ "@syntax" = "1" }
}
{ "config_br15"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
{ "bridge_br15"
{ "quote" = "\"" }
{ "value" = "vlan15" }
{ "@syntax" = "1" }
}
{ "config_vlan15"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
{ "vlan_start_eth1"
{ "quote" = "\"" }
{ "value" = "no" }
}
{ "RC_NEED_vlan11"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "RC_NEED_vlan10"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "RC_NEED_vlan15"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
test GentooConf_Net.lns get complicated_bridge_vlan =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "null" }
}
{ "config_eth1"
{ "quote" = "\"" }
{ "value" = "null" }
}
{ "vconfig_eth1"
{ "quote" = "\"" }
{ "value" = "set_name_type VLAN_PLUS_VID_NO_PAD" }
}
{ "vlans_eth1"
{ "quote" = "\"" }
{ "value" = "10 11 12" }
}
{ "config_vlan10"
{ "quote" = "\"" }
{ "value" = "null" }
}
{ "bridge_br10"
{ "quote" = "\"" }
{ "value" = "vlan10" }
}
{ "routes_br10"
{ "quote" = "\"" }
{ "value" = "default via 192.168.0.254" }
}
{ "config_br10"
{ "quote" = "\"" }
{ "value" = "192.168.0.1/22" }
}
{ "RC_NEED_br10"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "brctl_br10"
{ "quote" = "\"" }
{ "value" = "setfd 0" }
}
{ "bridge_br11"
{ "quote" = "\"" }
{ "value" = "vlan11" }
}
{ "config_vlan11"
{ "quote" = "\"" }
{ "value" = "null" }
}
{ "brctl_br11"
{ "quote" = "\"" }
{ "value" = "setfd 0" }
}
{ "RC_NEED_br11"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "config_br11"
{ "quote" = "\"" }
{ "value" = "null" }
}
{ "RC_NEED_br12"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "brctl_br12"
{ "quote" = "\"" }
{ "value" = "setfd 0" }
}
{ "config_br12"
{ "quote" = "\"" }
{ "value" = "null" }
}
{ "bridge_br12"
{ "quote" = "\"" }
{ "value" = "vlan12" }
}
{ "config_vlan12"
{ "quote" = "\"" }
{ "value" = "null" }
}
{ "vlan_start_eth1"
{ "quote" = "\"" }
{ "value" = "no" }
}
{ "RC_NEED_vlan11"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "RC_NEED_vlan10"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
{ "RC_NEED_vlan12"
{ "quote" = "\"" }
{ "value" = "net.eth1" }
}
test GentooConf_Net.lns get "config_eth0=\"null\"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "null" }
}
test GentooConf_Net.lns get "config_eth0=(\"null\")\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
test GentooConf_Net.lns get "config_eth0=(\"192.168.0.1/24\"
\"192.168.0.2/24\")\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "192.168.0.1/24" }
{ "value" = "192.168.0.2/24" }
{ "@syntax" = "1" }
}
test GentooConf_Net.lns put "" after
set "config_eth0/quote" "\"";
set "config_eth0/value" "192.168.0.1/24";
set "config_eth0/@syntax" "1"
= "config_eth0=(\"192.168.0.1/24\")\n"
test GentooConf_Net.lns put "" after
set "config_eth0/quote" "\"";
set "config_eth0/value[1]" "192.168.0.1/24";
set "config_eth0/value[2]" "192.168.0.2/24";
set "config_eth0/@syntax" "1"
= "config_eth0=(\"192.168.0.1/24\" \"192.168.0.2/24\")\n"
test GentooConf_Net.lns get "config_eth0=( \"null\" )\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "null" }
{ "@syntax" = "1" }
}
test GentooConf_Net.lns put "" after
set "config_eth0/quote" "\"";
set "config_eth0/value[1]" "null"
= "config_eth0=\"null\"\n"
(* empty space should get deleted *)
test GentooConf_Net.lns get "config_eth0=\" \"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = " " }
}
(* empty space should get deleted *)
test GentooConf_Net.lns get "config_eth0=\" \"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = " " }
}
test GentooConf_Net.lns get "vlans_eth1=\"10\"\n" =
{ "vlans_eth1"
{ "quote" = "\"" }
{ "value" = "10" }
}
test GentooConf_Net.lns put "" after
set "vlans_eth1/quote" "\"";
set "vlans_eth1/value[1]" "10"
= "vlans_eth1=\"10\"\n"
(* leading space should get deleted *)
test GentooConf_Net.lns get "vlans_eth1=\" 10\"\n" =
{ "vlans_eth1"
{ "quote" = "\"" }
{ "value" = " 10" }
}
(* trailing space should get deleted *)
test GentooConf_Net.lns get "vlans_eth1=\"10 \"\n" =
{ "vlans_eth1"
{ "quote" = "\"" }
{ "value" = "10 " }
}
test GentooConf_Net.lns get "vlans_eth1=\"10 11\"\n" =
{ "vlans_eth1"
{ "quote" = "\"" }
{ "value" = "10 11" }
}
test GentooConf_Net.lns put "" after
set "vlans_eth1/quote" "\"";
set "vlans_eth1/value[1]" "10 11"
= "vlans_eth1=\"10 11\"\n"
test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "192.168.0.1/24" }
}
test GentooConf_Net.lns put "" after
set "config_eth0/quote" "\"";
set "config_eth0/value[1]" "192.168.0.1/24"
= "config_eth0=\"192.168.0.1/24\"\n"
test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\n"
. "192.168.0.2/24\"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "192.168.0.1/24" }
{ "value" = "192.168.0.2/24" }
}
(* FIXME doesn't parse at all b/c of trailing newline *)
(* hard to fix; this might just be invalid *)
(*
test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\n"
. "192.168.0.2/24\n\"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "192.168.0.1/24" }
{ "value" = "192.168.0.2/24" }
}
*)
test GentooConf_Net.lns put "" after
set "config_eth0/quote" "\"";
set "config_eth0/value[1]" "192.168.0.1/24";
set "config_eth0/value[2]" "192.168.0.2/24"
= "config_eth0=\"192.168.0.1/24\n192.168.0.2/24\"\n"
test GentooConf_Net.lns get "config_eth0=\"192.168.0.2/24 scope host\"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "192.168.0.2/24 scope host" }
}
test GentooConf_Net.lns get "config_eth0=\"192.168.0.2/24 scope host\n"
. "4321:0:1:2:3:4:567:89ab/64 nodad home preferred_lft 0\"\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "192.168.0.2/24 scope host" }
{ "value" = "4321:0:1:2:3:4:567:89ab/64 nodad home preferred_lft 0" }
}
test GentooConf_Net.lns put "config_eth0=(\"192.168.0.1/24\")\n" after
rm "config_eth0/@syntax"
= "config_eth0=\"192.168.0.1/24\"\n"
test GentooConf_Net.lns put "config_eth0=(\"192.168.0.1/24\"
\"192.168.0.2/24\")\n" after
rm "config_eth0/@syntax"
= "config_eth0=\"192.168.0.1/24\n192.168.0.2/24\"\n"
test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\"\n"
. ". /etc/conf.d/net.extra\n" =
{ "config_eth0"
{ "quote" = "\"" }
{ "value" = "192.168.0.1/24" }
}
{ ".source" = "/etc/conf.d/net.extra" }
(*
config_eth0="null"
config_eth1="null"
vlans_eth1="10 11"
vlan10_name="vlan10"
config_vlan10="192.168.0.1/24"
routes_vlan10="10.0.0.0/8 via 192.168.0.2"
vlan11_name="vlan11"
config_vlan11="172.16.0.1/25"
routes_vlan11="default via 172.16.0.2"
#vlan_start_eth1="no"
#rc_need_vlan10="net.eth1"
#rc_need_vlan11="net.eth1"
*)
signature.asc
Description: OpenPGP digital signature
_______________________________________________ augeas-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/augeas-devel
