Server Fault
Server Fault

Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster. Ceph's main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-available.

223 questions
19
votes
1 answer

What is the difference between Ceph and NFS when it comes to shared file storage?

Can someone explain to me what is Ceph compared to NFS? From a shared file storage perspective? What is the added value of using Ceph instead of NFS?
Basil A
  • 2,300
18
votes
2 answers

Cause of page fragmentation on "large" server with xfs, 20 disks and Ceph

Any insight from someone with a bit of experience in the linux IO system would be helpful. Here is my story: Recently brought up a cluster of six Dell PowerEdge rx720xds to serve files via Ceph. These machines have 24 cores over two sockets with two…
pingu
  • 181
9
votes
3 answers

Is Ceph possible to handle hardware RAID arrays (LUNs) as OSD drives?

I am pretty new to Ceph and try to find out if Ceph supports hardware level raid HBAs. Sadly could not find any information. What I found is, that it is recommended to use plain disks for OSD. But this pushes the requirements to the PCIe, the…
cilap
  • 297
8
votes
2 answers

CEPH's raw space usage

I can't understand ceph raw space usage. I have 14 HDD (14 OSD's) on 7 servers , 3TB each HDD ~ 42 TB raw space in total. ceph -s osdmap e4055: 14 osds: 14 up, 14 in pgmap v8073416: 1920 pgs, 6 pools, 16777 GB data, 4196 kobjects …
virgism
  • 83
7
votes
1 answer

Better performance when HDD write cache is disabled? (HGST Ultrastar 7K6000 and Media Cache behavior)

Attention, please. Long read. During initial performance tests of Hitachi Ultrastar 7K6000 drives that I'm planning to use in my Ceph setup I've noticed a strange thing: write performance is better when disk write cache is disabled. I use…
J''
  • 91
5
votes
2 answers

Is ceph replication based on nodes or disks?

Im currently evaluating storage systems for xenserver. Because data replication is important in the case of a failure i have a question regarding replication in ceph. As far as i know every disk in a node is an osd by itself (disks are not in any…
laubed
  • 66
5
votes
1 answer

Why is GlusterFS so slow here?

We've set up a mirroring pair of GlusterFS servers. No special tuning, whatever came "out of the box" with GlusterFS-3.5.1 in the official RHEL6 RPM, that's what we have. The cluster works, but the performance is pretty awful. For example,…
Mikhail T.
  • 2,405
5
votes
1 answer

How can I measure complex I/O activity to develop realistic benchmarking?

I need to retrieve a few metric from my service server to benchmark another stroage solution. There are various options are provied by benchmark program(I choosed FIO,http://freecode.com/projects/fio). I could easily set up io pattern with it. But I…
jinhwan
  • 183
5
votes
1 answer

Ceph e5 handle_auth_request failed to assign global_id after a host outage

I have a small 3-host Ceph cluster with Ubuntu 20.04.1 and Ceph 15.2.5 using docker containers and deployed with cephadm. Yesterday one of the hosts (s65-ceph) had a power outage. The other two hosts continued working for a while but then s63-ceph…
Paolo Celati
  • 111
4
votes
1 answer

High availability for ceph storage?

Do ceph have high availability, I config 2 node like this cluster: id: 07df97db-f315-4c78-9d2a-ab85007a1856 health: HEALTH_WARN Reduced data availability: 32 pgs inactive Degraded data redundancy: 374/590…
Lê Minh Quân
  • 301
4
votes
1 answer

Which iscsi target on linux for serving vSphere?

All such questions are a bit dated; I'm hoping things have changed in the past year. What do you guys use to serve a DRS cluster with vSphere 5.1/5.5? SCSI-3 persistent reservation support would be definitely needed, VAAI is a nice-to-have. I've…
Zoltán
  • 155
4
votes
3 answers

Does it work to run mysql with the data directory on a CephFS mount?

We have a database cluster that uses MySQL on DRBD with Pacemaker+Corosync, which is great. My question is whether it is possible (and whether anyone has been successful) to mount a CephFS drive to /var/lib/mysql and run MySQL from there. The…
Bryan Agee
  • 1,229
4
votes
0 answers

Cannot Access Ceph NVMe-oF Device on Linux or Windows Client (Ceph 19, Debian 12, Hyper-V VMs)

I'm setting up a new Ceph cluster for testing purposes using virtual machines on Hyper-V, all running Debian 12 (Bookworm). Here's the current setup: 4 VMs, each with: 40GB disk for the system 1000GB disk for Ceph (clean, no partitions or…
Jan Wipp
  • 41
4
votes
3 answers

Ceph does not detect disks

I am using cephadm. I had a test cluster up and running. I re-installed the OS and during installation nuked the drives by deleting the LVM volume groups and partitions. Now lsblk shows the devices to be present but ceph orch device ls returns…
Tintin
  • 205
4
votes
0 answers

How to create Ceph OSDs on Partitions using cephadm, on ceph-pacific?

In the pacific version of Ceph, OSD creation is not allowing by cephadm on partitions.(We have NvMes with partitions). The command used is: ceph orch daemon add osd : The above command only accepts, disks or lvm, and throws error…
Arun Vinod
  • 51
1
2 3
…
14 15