28

Just a quick question, but why are there so many file systems still competing and in use today? (ntfs, fat32, ext3(ffs), etc)

It seems that file system designers could agree upon the best aspects of each type of system and implement a "best" filesystem, no? Just a thought, since these filesystems have been around for a while now, and it should be at least somewhat apparent which ones have good qualities over others, and we could just combine the good in each and create an ultimate system that is much better

Jungle Hunter
  • 2,515
  • 3
  • 25
  • 36
Dark Templar
  • 6,323

11 Answers11

32

Let's think about the specifics here for a moment, using examples you've cited:

  • ntfs - Proprietary to Microsoft. Anyone who is not Microsoft cannot use this, therefore would have to use/create something different. Now, if you are Microsoft, you want to use this over FAT because of the issues of the next bullet point.

  • fat32 - Not sufficiently modern. The maximum file size is 4GB. Directory entry lookup is O(n). The allocation table is a linked list, rather than something more efficient like an allocation bitmap (where it's really quick to find contiguous free space). Does not support permissions. Does not support hard links or symbolic links. Does not support journaling.

  • ext3 - This was an extension of ext2 mainly to support journaling.

So, it seems there are a few reasons:

  1. An earlier filesystem lacks something. In the case of FAT it is lacking a lot: both in terms of (1) features and (2) performance. In the case of ext2 it did not have journalled updates, so recovering from a crash took more time.

  2. An existing filesystem would probably do, but it is not yours. (eg. NTFS if you're not Microsoft). In this case you don't really have much choice but to come up with your own.

asveikau
  • 434
24

Short answer: One size does not fit all.

There are trade-offs. For example if you want a journaled FS, you pay (efficiency, complexity, etc.) for it but get something out of it. Some don't feel the need for a journaled FS and don't want to pay for it, some do. Same with other "features" of the FS.

Jungle Hunter
  • 2,515
  • 3
  • 25
  • 36
14

There can't ever be one "best" of anything because there are so many opinions about what "best" is. The decision is specific to the needs and limitations of the user. Designs are always based on their ability to fit within constraints.

A basic mobile phone needs to store a few hundred contacts, text message history, and a few small apps. Does its filesystem need to support a hierarchical directory structure on multi-terabyte drives in a RAID configuration? Is there sufficient RAM on the device to run such a filesystem? Does the filesystem need complex ACLs? Probably not -- for all of these questions -- so a simple, resource-sipping filesystem would suffice.

Companies will also develop different products to maintain a competitive advantage. For instance, Apple touts the ability of its HFS+ filesystem to track which files have changed recently so that backups are quick. On the flip side, the drivers for a floppy disk filesystem (FAT) can fit in just a few KB of memory.

Barry Brown
  • 4,095
11

Too much depends on what you want to optimize.

Consider FAT for a moment: its support for long file names is kludgy (to put it nicely), and searching through files in a directory is linear so it gets slow very quickly if a directory contains a lot of files. At the same time, it has a bare minimum of metadata to raw write speed is very good, and since it's so simple over all, code to implement it can be quite small.

Something like ext2 or ext3 adds a lot of features and capabilities that are simply absent from FAT. Searching for files is also quite a lot faster. At the same time, raw writing speed is probably a bit slower, and the code to implement the file system is undoubtedly quite a lot larger.

Jerry Coffin
  • 44,795
9

Just a quick question, but why are there so many file systems still competing and in use today? (ntfs, fat32, ext3(ffs), etc)

It seems that file system designers could agree upon the best aspects of each type of system and implement a "best" filesystem, no?

Let's suppose that there weren't tradeoffs, and file system designers did implement a "best" filesystem, free from patent worries, and released as dual-licence BSD/GPL so that it would be acceptable to MS and Debian alike. What makes you think that the other file systems would vanish overnight?

I don't think anyone's used FAT32 on a new hard drive for 10 years, but it still persists as the de facto standard for formatting USB drives, SD cards, etc. Camera and mobile phone manufacturers have tried and tested firmware for using it. Arduino hobbyists have stable libraries for using it. They're all going to need big incentives to change.

And then you have the issues of backwards compatibility with older OSes (especially Windows, whose users won't want to install new filesystem drivers).

Peter Taylor
  • 4,043
4

As is so often the case in computing the answer is (a) because of historical circumstance and the need to maintain backwards compatibility and (b) because some methods are better suited to some tasks than others.

On (a) you need to remember that the "Winchester drive" - I am just about old enough to remember them being called that - (what the rest of the world calls a 'hard drive') has only been around for about half the time of electronic computing and even then it has not been accessible to most users for even that long for cost reasons. The FAT file system worked well on floppy disks and also on the original small hard drives as it was reasonably efficient and required low overhead. Once it started to be used - and its use spread widely because it is simple to implement - manufacturers could not tell its users that their old data was suddenly invalid.

Similarly, for Linux users, say, a stable NTFS driver was a long time coming, so keeping devices formatted as FAT meant they could be read and written across multiple systems.

On (b) - think of the differences between a system that, say, stores, billions of text-based database records and one that stores DVD-length media files. For the databse each record could be very small - perhaps just 30 or 40 bytes and certainly a filesystem that allocated a whole 'segment' (however you want to define that) of disk is likely to be wasteful of diskspace. Not so with the DVDs - bigger 'segments' (within reason, obviously) are likely to be highly efficient in space terms.

So different filesystems are designed for different purposes.

3

Yet another example of why there can never be a perfect file system for everyone: HDDs and SSDs have very different read/write access characteristics. An SSD-optimized filesystem would probably work best by fragmenting files like crazy but with every fragment the page size of the SSD itself; this would perform terribly on an HDD. An HDD-optimized filesystem tries to keep files as unfragmented as possible and even puts frequently-used files in the "hot" area on the faster-spinning outer portion of the platter; these characteristics wouldn't help SSD read speeds at all, and puts a huge encumbrance on how they are written to.

fluffy
  • 410
2

I think a very important fact is missing. Most of the time programmers tend to think that their way of doing something is superior to all other ways and therefore they look at a File System design and come up with problems and solutions that seem to be more general, elegant, fast, right... And if this feeling is strong enough they are free to build their own file system.

This leeds to competition, fragmentation, confustion and I hope in the end better solutions and more options to choose a matching solution for you.

Janusz
  • 171
  • 1
  • 3
1

Here's another concrete example of why you would need a separate file system or extend the functionality of an existing FS.

  1. Say you are a database vendor and would like to reduce the complexity of managing striping / mirroring of data to improve IO. You will feel the need to extend the functionality of basic file systems as Oracle did with ASM (Automated Storage Manager) which is like a Logical Volume Manager.
RHT
  • 151
1

In your list you mention an old filesystem used because a better one is no available however it is fast.

There are other filesystems. I heard google filesystem is mostly for fast duplication/redundancy if one harddrive or server goes down. I remember hearing of another filesystem made for many small files and to be used on a system for lots of request on small file (thumbnails).

Essentially they have difference goals and or may be propriety vs open source.

0

I think that ZFS (use by Solaris in Sun system [now Oracle]) is THE solution for file system.

Unfortunatly Oracle close OpenSolaris for discoverer and test it.

ZFS is open source, some Linux are trying to integrate it, look in Wikipedia for more informations.

cl-r
  • 37