0

I was trying to take backup using oplog based on ts using below code 

 import pymongo
    import traceback
    from pymongo.cursor import CursorType
    c = pymongo.MongoClient('mongodb://localhost:28001')
    oplog = c.local.oplog.rs
first = next(oplog.find().sort('$natural', pymongo.DESCENDING).limit(-1))
ts = first['ts']
while True:
    cursor = oplog.find({'ts': {'$gt': ts}}, cursor_type=CursorType.TAILABLE_AWAIT, oplog_replay=True)
    while cursor.alive:
        for doc in cursor:
            ts = doc['ts']
            print doc['ns']
            db,collection=str.split(str(doc['ns']),'.',1)

It is working well when we have less opLog size but as i am using it on production where oplog size is 48 GB around it is not working.

I cannot alter oplog size.Please suggest some way to take incremental backup.

Md Haidar Ali Khan
  • 6,523
  • 9
  • 40
  • 62
Gagan Sidhu
  • 120
  • 7

2 Answers2

1

Incremental backup procedure

  • lock writes on a secondary member

  • Dump oplog from the recorded oplog position on full (or latest incremental ) backup:

    mongodump --host <secondary> -d local -c oplog.rs -o /mnt/mongo-test_backup/1 
--query '{ "ts" : { $gt :  Timestamp(1437725201, 50) } }'

  • Record latest oplog position (same way as for full backups)

  • Unlock writes

Click Here for detailed information

RDFozz
  • 11,731
  • 4
  • 25
  • 38
0

I'm thinking to do incremental backup using mongodump,like followings.

mongodump -d local -c oplog.rs -q "{ts : { \"\$gt\" : { \"\$timestamp\" : { \"t\" : 1494476525, \"i\" : 0 } } }}" --port 27022 --archive=/home/mongodb/testoplog.tar.gz --gzip >> testoplog.log 2>&1
Michael Green
  • 25,255
  • 13
  • 54
  • 100
Nakai Kanako
  • 1