LDAP: backup with slapcat vs ldapsearch

Question

Used: openldap-servers-2.4.23-34.el6_5.1.x86_64

Task: create script for crontab to create scheduled database full backup.

1. slapcat - create file in the default format, LDIF.

2. slapcat can be done while slapd running.

3. To restore file after slapcat - slapadd must be used (not ldapadd).

4. slapcat/add doesn't require password.

5. slapadd can be done only when slapd stopped (and corresponding database is empty).

Example:

 $ slapcat -f /etc/openldap/slapd.conf -b "dc=db_1" -l db_1_backup.ldif
 $ slapadd -l db_1_backup.ldif

Instead of slapcat/add - let's take a look at ldapsearch/add:

1. ldapsearch - creates file with almost same information as slapcat;

2. ldapadd - can use file from ldapsearch, doesn't require slapd to be stopped;

3. ldapadd/search - requires password.

Example:

 $ ldapsearch -D "cn=root,dc=db_1" -W -b "dc=db_1" "dc=db_1" -LLL > db_1_backup2.ldif
 $ ldapadd -x -D "cn=root,dc=db_1" -W -f db_1_backup2.ldif

So - The Question is:

1. Am I missing anything in this tools description?

2. What else is difference between ldapadd/slapadd and ladpsearch/slapcat?

Answer 1

Good summary, some additional points:

• slapcat dumps from whatever the (local) direct storage backend is, it need not be Berkeley (hdb or bdb), it also works with OLC (OpenLDAP Configuration) (cn=config). It dumps to LDIF format. (By direct I mean directly managed by OpenLDAP, not for example an SQL backend, even if its stored locally.)
• ldapadd requires that slapd is running, slapadd requires that it is not running
• ldapsearch requires that slapd is running, slapcat doesn't care if it's running with a BDB backend certain backends, as you noted (this is OpenLDAP version and backend dependent)

In short:

• slapcat is the way to get a good backup that you can restore quickly (albeit possibly with downtime on the master depending on backend(s) in use, though you can work around this with various types of replication set up). This is what you should use for a general backup, and pre-upgrade backups.
• for slapcat, some backends may require slapd is not running, others are safe with slapd running if your database is read-only, others are documented as being safe (since OpenLDAP-2.5 only the MDB and null backends are slapcat safe when slapd is running. This varies by version and backend, check your slapcat and slapd-<backendname> man pages.
• ldapsearch (without +) will get you a portable backup you can probably load with little difficulty into any other directory server, but it will only be a viable restore in a simple OpenLDAP set up (no replication, no special overlays, no rewriting), and if you don't care about preserving UUID/create/modify meta-data. You'll need any extra schema files that go along with your data too.
• ldapadd (using its other identity ldapmodify) can be used to easily apply LDAP modifications (object modify, delete and rename) which are not feasible or possible with slapadd/slapcat alone

For most admins the main considerations arise from the slightly different contents of the LDIF in each case, and the requirement for slapd to be running (or not). The more important differences are:

1. slapcat is faster because it simply dumps the database, skipping the LDAP protocol overheads, authentication, access control, object and time limits, overlays; and it does not search according to the LDAP hierarchy.
2. slapadd is faster (again, no LDAP protocol overheads), and in the case that you are restoring a known-good backup you can run in quick mode (-q) to speed up large imports. You can also disable schema checking (-s), though note than small changes in schema or data validation between OpenLDAP versions are not unheard of.
3. slapcat is limited to local databases, it will not cross over to other directories (e.g. with back-ldap, back-meta) the way ldapsearch will. The same applies to slapadd/ldapadd.
4. ldapsearch will return dynamic attributes which are not stored in a backend, e.g. hasSubordinates or those maintained by overlays (e.g. slapo-memberof). You will have problems loading these with ldapadd (e.g. operational attributes with no-user-modification). Rewriting (slapo-rwm) may also distort ldapsearch's view of the directory contents.
5. slapcat includes internal (operational) attributes, if you are using replication these attributes are critical. With replication then you are somewhat less dependent on backups, but if you use ldapadd to reload your master, every object will be recreated by replication (changed entryUUID entryCSN) Although you can include operational attributes by using the special "+" attribute with ldapsearch (or allop overlay), this is not the same thing as slapcat, see the previous point for why this is so. These attributes also include create/modify DN and timestamps, which may be important to some applications.
6. because slapcat does not observe the LDAP hierarchy (implied ordering), there's no guarantee that its data ordering is going to be viable with ldapadd - i.e. even if you strip out the operational attributes, ldapadd can complain because sub-ordinates may appear before their superiors (parents). The LDAP specs require a parent to exist, but also leave the search result ordering undefined in this respect. See Howard's comment below, OpenLDAP's slapadd silently supports unordered data for some backends. In a pinch you may be able to repeatedly use slapadd with the continue on error option (-c) until all the "out of order" parents are created, stopping when you no longer receive any error code 32 (no such object, meaning missing parent) and receive only code 68 (already exists) for every object.
7. ldapadd is subject to the LDAP rules and overlays, e.g. referential integrity, ppolicy (password policy)
8. slapcat prefers to use base-64 encoded attribute values for at least userPassword (indicated with :: after the attribute name)
9. ldapsearch has more options for LDIF formatting, and writing large attributes to separate files. It may also handle referrals and aliases.

Answer 2

slapcat does not work if you have overlays, e.g. memberOf. So if you do a slapcat / slapadd cycle membership overlays will no longer work.

Answer 3

slapcat/slapadd are logical data migration tools for LDAP very similar to logical dump and load of relational databases. They are very slow to create and to restore. Best way to backup full databases is to create a volume snapshot and backing up physical database files and the configuration folder (eg. /etc/ldap/slapd.d).

Creating LDIF files of individual records using ldapsearch is useful for archiving specific records and should not be used for regular full database backups.