Are my S3 ceph bucket synchronized or synchronizing?

Question

I have two pacific ceph cluster.

ceph --version
ceph version 16.2.10-160.el8cp (6977980612de1db28e41e0a90ff779627cde7a8c) pacific (stable)

(that is 5.3.2)

There is a dual replication set between the two sites (named gondor and rohan below).

I am new to ceph and replications, I started a synchronisation 3 days ago, using this command from rohan side:

radosgw-admin data sync init --bucket=bm.test.sync.bucket1 --source-zone=gondor

now on both side, every time I issue radosgw-admin sync status I get thoses two results:

from gondor

          realm UUID (middleearth)
      zonegroup UUID (thirdage)
           zone UUID (gondor)
   current time 2024-12-19T07:15:45Z
zonegroup features enabled:
                   disabled: resharding
  metadata sync no sync (zone is master)
      data sync source: UUID (rohan)
                        syncing
                        full sync: 0/128 shards
                        incremental sync: 128/128 shards
                        data is behind on 3 shards
                        behind shards: [13,17,52]
                        oldest incremental change not applied: 2024-12-19T08:15:32.613647+0100 [13]

from rohan

          realm UUID (middleearth)
      zonegroup UUID (thirdage)
           zone UUID (rohan)
   current time 2024-12-19T07:34:55Z
zonegroup features enabled:
                   disabled: resharding
  metadata sync syncing
                full sync: 0/64 shards
                incremental sync: 64/64 shards
                metadata is caught up with master
      data sync source: UUID (gondor)
                        syncing
                        full sync: 0/128 shards
                        incremental sync: 128/128 shards
                        123 shards are recovering
                        recovering shards: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127]

Questions :

are my cluster synching ? (that is, will I get less "recovering shards" at some point ?)

I can understand there is a replication latency and a little discrepencies on most recents objects.

However, my buckets don't seems to have same contents on both side.

In one particular bucket

on gondor side : two objects : one from june 2022, one empty from january 2024
on rohan side only one empty from january 2024

I tought above command would resynchronize bucket. (radosgw-admin data sync init --bucket=bm.test.sync.bucket1 --source-zone=gondor)

Can sync be achieved ? If yes, how ?

edit 1:

typos in data sync, was data sync init, not ~~data sync status~~

Are my S3 ceph bucket synchronized or synchronizing?

0 Answers0