Server Fault
Server Fault

Questions tagged [nvme]

132 questions
26
votes
2 answers

Why is there both character device and block device for nvme?

I am trying to understand the configuration of nvme. But I do not understand why there are two devices: nvme block and nvme character device: crw------- 1 root root 243, 0 Dec 12 16:09 /dev/nvme0 brw-rw---- 1 root disk 259, 0 Jan 14 01:30…
GP92
  • 781
17
votes
8 answers

Understanding NVMe storage and hardware requirements

I'm a bit confused about the recent developments in PCIe-based storage, particularly as it relates to the NVMe specification and its hardware compatibility. While I've worked extensively with SSDs in disk form-factor and some higher-end PCIe devices…
ewwhite
  • 201,205
12
votes
3 answers

Automatically mount SSD instance storage on AWS EC2 in Ubuntu 16.04

I am using an Ubuntu 16.04 EC2 from AWS on c5d.2xlarge HW. It comes with a 200 GB SSD called /dev/nvme1n1. I am able to mount this drive using: $ sudo mkfs.ext4 -E nodiscard /dev/nvme1n1 $ sudo mount -o discard /dev/nvme1n1 /home/ubuntu To try…
John Corson
  • 123
9
votes
1 answer

Understanding ZFS NVMe benchmarks with FIO

I'm trying to benchmark an all-NVMe ZFS disk array. I'm familiar with extremely fast benchmark results, with very little disk activity due to efficient ZFS caching. I ran into the reverse: massive amount of ZFS disk activity, but FIO shows only…
Frank Vermeulen
  • 167
8
votes
1 answer

The setup of S2D that delivers up to 2M IOPS to SQL FCI

We are about to deploy shared storage researching for ultra-fast storage to implement Microsoft SQL Server Failover Cluster (FCI). So far the project goes, we would to start with 500K IOPS for 8k blocks about 70r/30w pattern. Also we would like to…
Joshua Turnwell
  • 530
8
votes
3 answers

Unexpected poor performance with NVMe drives on an X9DR3-F

I am experiencing what seems like uncharacteristically low performance from an NVMe SSD stripe in a server. The hardware is as follows: Motherboard: X9DR3-F CPU: Dual E5-2650v2 RAM: 128GB DDR3-1333 UDIMM (16x8GB) NVMe drives: 4x…
Bryan B
  • 379
7
votes
2 answers

Does any PCIe slot support NVMe SSD's automatically or do I need a specific M.2 slot?

I have been trying to dissect the information I find with googling, but have come to the conclusion that I cannot find with certainty the answer to the following questions: Given you know the basics of your own desktop system, how do I know it…
Abel
  • 1,077
7
votes
3 answers

MongoDB on NVMe in production

I'm setting up my db production environment and I have some questions. The server has two NVMe disks, but they cannot be used in the hardware RAID controller. Do I need RAID with NVMe? Would it be enough to use them as JBOD in production? Or should…
Green
  • 173
7
votes
0 answers

Write performance is 5 times worse with LUKS on top of mdadm RAID10 than without LUKS

I have servers with many NVMe disks. I am testing disk performance with fio using the following: fio --name=asdf --rw=randwrite --direct=1 --ioengine=libaio --bs=16k --numjobs=8 --size=10G --runtime=60 --group_reporting For a single disk, LUKS…
tacos_tacos_tacos
  • 3,260
6
votes
3 answers

How to limit ZFS writes on NVME SSD in RAID1 to avoid rapid disk wear?

Currently I'm running Proxmox 5.3-7 on ZFS with few idling debian virtual machines. I'm using two SSDPE2MX450G7 NVME drives in RAID 1. After 245 days of running this setup the S.M.A.R.T values are terrible. SMART/Health Information (NVMe Log 0x02,…
Ahupathi R.
  • 63
6
votes
3 answers

Disabling ext4 write barriers when using an external journal

I'm currently experimenting with different ways of improving write speeds to a fairly large, rotating disk-based, software-raid (mdadm) array on Debian using fast NVMe devices. I found that using a pair of such devices (raid1, mirrored) to store the…
jcharaoui
  • 332
6
votes
3 answers

What is the Long term stability of NVMe or SSD stored data?

Background: We need ready access to 30TB of audio data, although only a small fraction of it is ever requested for playback, that playback needs to be done immediately even for multi-year old data. The data resides in a SAN of multiple arrays and a…
Rowan Hawkins
  • 640
6
votes
2 answers

Supermicro Server fails to boot from NVMe SSD

I have a SuperMicro server with an NVMe boot drive and an additional SSD for data. I was attempting to install ubuntu on the NVMe boot drive and even though the installation succeeds and the correct boot device is set in the BIOS, the server does…
Keno Fischer
  • 271
6
votes
2 answers

NVMe parallel writes

I'm trying to understand better how writing to NVMe can be optimized. I have a process which writes a large amount of data to disk (~100 gb) in one batch job. The data is spread across 100s of files. I have multiple NVMe disks available and will be…
R.Smith
  • 61
6
votes
0 answers

AWS: Can't mount my restored volume (EBS snapshot)

I restored an EBS volume from AWS Backup and attached it to a new EC2 instance. When I lsblk I can see it under the name /dev/nvme1n1. More specifically the output of lsblk is: NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 …
HelloWorld
  • 157
1
2 3
…
8 9