1

We have space problems. The actual data size is only 5 GB, but disk size is 15 GB because of deleted documents. MongoDB (3.0.6) runs in docker container where we can't extend volume.

# tail -f mongod.log
2016-04-26T14:43:59.631+0000 I STORAGE  [FileAllocator] allocating new datafile /data/work/mongodb/data/opWNFbyNlGQ6xShT.11, filling with zeroes...
2016-04-26T14:43:59.632+0000 I STORAGE  [FileAllocator] FileAllocator: posix_fallocate failed: errno:28 No space left on device falling back
2016-04-26T14:43:59.632+0000 I STORAGE  [FileAllocator] error: failed to allocate new file: /data/work/mongodb/data/opWNFbyNlGQ6xShT.11 size: 2146435072 failure creating new datafile; lseek failed for fd 40 with errno: errno:2 No s^C

disk

/dev/vdh                            16G   15G  553M  97% /data/a6a186a3-dc98-4f47-83f3-96e953c18b69

repairDatabase doesn't work because no scratch space

> db.runCommand( { repairDatabase: 1 } );
{
        "ok" : 0,
        "errmsg" : "Cannot repair database opWNFbyNlGQ6xShT having size: 14975762432 (bytes) because free disk space is: 578928640 (bytes)",
        "code" : 14031
}

So I tried

> db.runCommand ( { compact: 'accounts', paddingFactor: 1.1 } )
{ "ok" : 1 }

I don't see nothing in progress

> db.currentOp()
{ "inprog" : [ ] }
> db.currentOp()
{ "inprog" : [ ] }
> db.currentOp()
{ "inprog" : [ ] }

There is no entry in mongodb.log about the compact operation

You may view the intermediate progress either by viewing the mongod log file or by running the db.currentOp() in another shell instance.

compact

Why compact fails?

is our only solution to do dump/drop/restore?

Sybil
  • 2,578
  • 6
  • 34
  • 61

1 Answers1

4

Since you are using the MMAP storage engine there are several factors to be aware of:

  • The compact command only defragments data files & indexes in MMAP; it does not release unused space to the operating system. Running compact can still be useful to reduce fragmentation and encourage free space reuse, but will not help if your disk space is critically low.

  • The compact command in MMAP requires additional disk space (up to 2GB) during the compact operation.

  • As at MongoDB 3.2, compact is a blocking command (database-level lock for the collection being compacted).

The approach to reclaiming preallocated space in MMAPv1 is rebuilding the database either through a resync (recommended for replica sets) or a repair (the only option for standalone deployments).

repairDatabase doesn't work because no scratch space

You can use the command line option --repairpath or the config file option storage.repairPath to provide extra storage (i.e. a second volume) for working space during the repair process:

mongod --repair --repairpath /path/to/extraspace --dbpath ...

After the repair process completes, the rebuilt data files will reside in the dbpath and the repairpath should be empty.

WiredTiger Storage Engine

If you have the option of upgrading to the WiredTiger storage engine in future, it has many improvements over MMAP:

  • Data files and indexes are compressed by default (with configurable compression options that can be set globally or per collection).
  • The compact command can release storage to the O/S (MongoDB 3.2.3+).
  • Minimal additional space should be required in order to run the compact command.
Stennie
  • 10,345
  • 2
  • 31
  • 46