Extending Puppet (probably with augeas) for custom configuration format

Question

I have a custom firewall system with our own simple configuration file. Its basically bash source defining a well known variables:

SUPERACCESS="127.0.0.1 192.168.11.0/24"
SERVICES="ping 80/tcp 443/tcp 22/tcp"
FORWARDING=1
.....

and I would like to manage these configuration files with puppet (probably via augeas if needed). Because I am new to the whole puppet ecosystem, I am looking for "best practice" of reasonably simple and clean way how to achieve this.

I need to be able to modify only some of the options (so I can not send the config as whole file from puppet master) and I do not want to limit the access to these files exclusively to puppet. All I need is a way how to ensure that certain subnet, service, etc... is (or is not) in the appropriate list, i.e. something like this:

firewall::superaccess { "LAN" :
         target => "192.168.11.0/24",
          ensure=> "present",
}

I managed to force augeas into using Shell_variables lens and I was able to change the "simple" fields like FORWARDING. Unfortunatelly I did not succeed with the list values (i.e. SUPERACCESS) using the Shell_variables_list lens.

Maybe there is better way than using augeas. I tried to look on puppet forge if there is some module with similiar config format but I failed to find any. Unfortunately I do not know much of ruby and augeas lens language but I will learn some if needed. I just do not want to start in the wrong direction...

Update: Thanks to Raphink I managed to locate the problem. It is the comment in the middle of the line. If there is this line in the file:

service_ping="ping/icmp" #ping

I get this error:

/tmp/augcheck.TCTUBq/parse_file.aug:3.0-.58:exception thrown in test
/tmp/augcheck.TCTUBq/parse_file.aug:3.5-.54:exception: Iterated lens matched less than it should
    Lens: /usr/share/augeas/lenses/dist/shellvars_list.aug:40.12-.37:
    Error encountered at 176:0 (9906 characters into string)
    <--------\n#Built in services\n|=|service_ping="ping/icmp" #pi>

    Tree generated so far:

If I move the comment on separate line, it loads the file into the tree structure. Even augtool works as expected. I have these verison of augeas installed: ruby-augeas-0.4.1-3.fc17.x86_64, augeas-1.0.0-1.fc17.x86_64, augeas-libs-1.0.0-1.fc17.x86_64

Update 2: To solve the problem with inline comments I used code from ShellVars lens. The change is to copy/paste few definitions from ShellVars and change the definition of line:

  let empty_part_re = Util.empty_generic_re . /\n+/
  let semicol_eol = del (/[ \t]*[;\n]/ . empty_part_re*) "\n"
  let eol_for_comment = del /([ \t]*\n)([ \t]*(#[ \t]*)?\n)*/ "\n"
  (* comment_eol in shell MUST begin with a space *)
  let comment_eol = Util.comment_generic_seteol /[ \t]+#[ \t]*/ " # " eol_for_comment
  let comment_or_eol = comment_eol | semicol_eol

  let kv = [ key key_re . eq . ( squote_arr | dquote_arr | unquot_val ) .  comment_or_eol ]

score 3 · Accepted Answer · answered May 01 '13 at 20:40

Yes, the Shellvars_list lens should be able to parse your file as:

{ "SUPERACCESS"
  { "quote" = """ }
  { "value" = "127.0.0.1" }
  { "value" = "192.168.11.0/24" }
}
{ "SERVICES"
  { "quote" = """ }
  { "value" = "ping" }
  { "value" = "80/tcp" }
  { "value" = "443/tcp" }
  { "value" = "22/tcp" }
}
{ "FORWARDING"
  { "quote" = "" }
  { "value" = "1" }
}

Note: you can test that using augcheck:

augcheck /etc/firewall/config Shellvars_list

Now, the problem is: "How do you get from that tree to your definition?", that is:

firewall::superaccess { "LAN" :
         target => "192.168.11.0/24",
          ensure=> "present",
}

where you'd want to ensure the presence of one element in a list.

You could do:

define firewall::superaccess (
  $target,
  $ensure = 'present',
) {
  case $ensure {
    'present': {
      $changes = "set SUPERACCESS/value[.='${target}'] '${target}'"
    }

    'absent': {
      $changes = "rm SUPERACCESS/value[.='${target}']"
    }

    default: { fail "Wrong value for ensure: '${ensure}'" }
  }

  augeas { "Set target ${target} in /etc/firewall/config":
    lens    => 'Shellvars_list.lns',
    incl    => '/etc/firewall/config',
    changes => $changes,
  }
}

score 0 · Answer 2 · answered May 01 '13 at 19:39

What I would probably do is change your script so that the variables come from an external file, such as /etc/sysconfig/firewall-info or something (on say a redhat box or similar). Then you don't have to modify or manage the script at all, just the config file, and let the script read in the variables from the config file. Then you could easily manage the config file using the augeas shellvars lens, or even make a custom provider.

Extending Puppet (probably with augeas) for custom configuration format

2 Answers2