Linux Don’t Need No Stinkin’ ZFS: BTRFS Intro

AdsSpy:2 sites by this GAnalytics ID 
<a href="http://d1.openx.org/ck.php?oaparams=2__bannerid=74451__zoneid=11137__cb=e6928a9aae__oadest=http://ad.doubleclick.net/jump/N3271.linux-mag.com/B3613383.3;abr=!ie4;abr=!ie5;sz=640x480;ord=[timestamp]?" target="_blank"> <IMG SRC="http://ad.doubleclick.net/ad/N3271.linux-mag.com/B3613383.3;abr=!ie4;abr=!ie5;sz=640x480;ord=[timestamp]?" BORDER=0 WIDTH=640 HEIGHT=480 ALT="Click Here"></A>
Tech Problems?

<SCRIPT language='JavaScript1.1' SRC="http://ad.doubleclick.net/adj/N815.linuxmagazine.com/B3523757;click0=http://www.gadex.com/www/delivery/ck.php?oaparams=2__bannerid=174__zoneid=9__cb=e0e1327eab__maxdest=;abr=!ie;sz=336x280;ord=e0e1327eab?"> </SCRIPT> <NOSCRIPT> <a href="http://www.gadex.com/www/delivery/ck.php?oaparams=2__bannerid=174__zoneid=9__cb=e0e1327eab__maxdest=http://ad.doubleclick.net/jump/N815.linuxmagazine.com/B3523757;abr=!ie4;abr=!ie5;sz=336x280;ord=e0e1327eab?" target="_blank"> <IMG SRC="http://ad.doubleclick.net/ad/N815.linuxmagazine.com/B3523757;abr=!ie4;abr=!ie5;sz=336x280;ord=e0e1327eab?" BORDER=0 WIDTH=336 HEIGHT=280 ALT="Click Here"></A> </NOSCRIPT>
x
Loading

Looking for updates? Follow Linux Magazine onTwitter. Hello, Guest |Login |Register

Analysis
Interviews
Webinars
White Papers
Case Studies
Blades
Desktops
Storage
Networking
Virtualization
Distros
Audio & Video
Virtualization
Storage
Databases
Networking
Security
HPC
Finance
Healthcare
Education
Government
Retail
Telecom
Web Development
Multicore
Desktop
Mobile
Kernel
Back Issues
Linux Don't Need No Stinkin' ZFS: BTRFS Intro & Benchmarks
ZFS may be locked into the Solaris operating system but “Butter FS” is on the horizon and it’s boasting more features and better performance.
Jeff Layton
Tuesday, April 21st, 2009
Community Tools

Recommend This [?]
(15 votes, average: 4.2 out of 5)
 Loading ...
Users Who Liked This [?]


Tags:linux btrfs

 22 Comments(view all)
<SCRIPT language='JavaScript1.1' SRC="http://ad.doubleclick.net/adj/N815.linuxmagazine.com/B3523757.3;click0=http://d1.openx.org/ck.php?oaparams=2__bannerid=74508__zoneid=17129__cb=9535b8e2b4__oadest=;abr=!ie;sz=300x250;ord=9535b8e2b4?"> </SCRIPT> <NOSCRIPT> <a href="http://d1.openx.org/ck.php?oaparams=2__bannerid=74508__zoneid=17129__cb=9535b8e2b4__oadest=http://ad.doubleclick.net/jump/N815.linuxmagazine.com/B3523757.3;abr=!ie4;abr=!ie5;sz=300x250;ord=9535b8e2b4?" target="_blank"> <IMG SRC="http://ad.doubleclick.net/ad/N815.linuxmagazine.com/B3523757.3;abr=!ie4;abr=!ie5;sz=300x250;ord=9535b8e2b4?" BORDER=0 WIDTH=300 HEIGHT=250 ALT="Click Here"></A> </NOSCRIPT>
Linux is a fun project to watch because you get to see this living and breathing piece of software change and evolve.
Recently, Linux has been undergoing something of a revolution with a respect to file systems. There are a number of projects includingext4 (Seeext4 File System: Introduction and Benchmarks),squashfs,nilfs which have made it into the kernel, andtux3, which hasn’t made it into the kernel yet but is under heavy development.
And perhaps most importantly,btrfs. Btrfs holds the promise of giving Linux many enterprise class file system features similar to ZFS (SeeSolving Common Administration Problems with ZFS andZFS on FUSE) but with even more features and better performance. In fact, many Linux experts think that btrfs is one of the keys to the future of Linux. While btrfs is not quite ready to be your only file system, it is in the kernel ready for testing and is still undergoing very heavy development.
In this article we will introduce the key features of btrfs find out how it compares to existing file systems. We will also test it usingiozone, with the understanding that it’s still in development so performance will likely change over time.
Introduction to btrfs
Btrfs, sometimes called “Butter FS”, is a new file system for Linux that has a wealth of features users have been wanting for some time.Chris Mason, who works atOracle, started the project and Oracle has released the project under the GPL license so anyone can work on it. Btrfs has a gained a very large following within the Linux community and has whipped up a good deal of buzz. Some of thefeatures of btrfs that have led to this buzz are:
Copy on Write
Extents (see article aboutext4)
Space efficient packing of small files (file systems have needed this for a long time)
Space efficient indexed directories
Dynamic inode allocation
Writable snapshots
Snapshots of snapshots
Subvolumes (separate internal filesystem roots)
Object level mirroring and striping
Checksums on data and metadata
Compression
Integrated multiple device support, with several RAID algorithms
Online filesystem check and defragmentation
Very fast offline filesystem check
Efficient incremental backup and File System mirroring
You can tell from the list of features that btrfs is a very ambitious project with a load of features (got to love that).
One of the most immediate questions one would have about btrfs is, “how does it compare to ZFS?” That’s a great question. There is a pretty goodblog posting that attempts to impartially compare btrfs and ZFS. While the comparison is quite long, a quick summary of the comparison is below:
Feature ZFS BTRFS
Copy on Write yes yes
Snapshots yes yes
Snapshots of snapshots ? yes
Performance degradation at near 98-100% disk usage yes Most likely yes
Block level compression yes Currently a mount option
Disk encryption Being developed Planned but not currently in kernel. Encryptfs could be an option
Online resizing no (being worked on) yes
Online defragmentation no (being being worked on as part of other projects) yes
Write checksums yes yes
Built-in RAID yes (0,1,10, 5 with fixes for write hole (raidz), and some variant of 6 called raidz) yes (0, 1, 10)
ACL yes yes
Direct IO yes Writes yes. Reads No - Planned
Quotas yes yes
Next: Feature Breakdown
1234Next »
Read More
ext4 File System: Introduction and Benchmarks
ZFS on FUSE
Solving Common Administration Problems with ZFS
Getting Ready for Samba4
Parallel Platters: File Systems for HPC Clusters Part Three

22 Comments on Linux Don't Need No Stinkin' ZFS: BTRFS Intro & Benchmarks»
Comments via RSS
<SCRIPT language='JavaScript1.1' SRC="http://ad.doubleclick.net/adj/N815.linuxmagazine.com/B3523757.4;click0=http://d1.openx.org/ck.php?oaparams=2__bannerid=74509__zoneid=29577__cb=4bfa2d7916__oadest=;abr=!ie;sz=160x600;ord=1242017750.8481?"> </SCRIPT> <NOSCRIPT> <a href="http://d1.openx.org/ck.php?oaparams=2__bannerid=74509__zoneid=29577__cb=4bfa2d7916__oadest=http://ad.doubleclick.net/jump/N815.linuxmagazine.com/B3523757.4;abr=!ie4;abr=!ie5;sz=160x600;ord=1242017750.8481?" target="_blank"> <IMG SRC="http://ad.doubleclick.net/ad/N815.linuxmagazine.com/B3523757.4;abr=!ie4;abr=!ie5;sz=160x600;ord=1242017750.8481?" BORDER=0 WIDTH=160 HEIGHT=600 ALT="Click Here"></A> </NOSCRIPT>
waddles said: +1   
Jeff - you didn’t mention what type of disks you used for benchmarking, but your results sure look like you’re hitting the practicalbandwidth limits of SATA-150. I sure would like to see a benchmark on InfiniBand 12x.
Also, what happened to comparing Ext3/Ext4 on RAID?
April 22nd, 2009 9:49 AM (permalink)
Log in to Reply
laytonjb said: +0   
Yep - I’m hitting the bandwidth limit pretty quickly. It’s a pretty old machine. I’m trying to get the family funding agency to fund my proposal for a new machine. :) No word yet but I think the funding agency is waiting for their stimulus package to arrive.
Are you looking for a benchmark of a system with a number of drives and btrfs exporting it over NFS/RDMA over IB?
April 22nd, 2009 2:07 PM (permalink)
Log in to Reply
Trackback:THEBIGIRON.COM » Blog Archive » BTRFS vs. ZFS
samrawlins said: -1   
Jeff, great article! Very informative summaries of the new features. I hope you write a follow-up in a few months (years?) when BTRFS is declared stable. What type of disk(s) are you using?
I do take issue with your claims regarding compression:
When compression was turned on the performance increased by a large amount. The write and rewrite performance almost doubles and the read and reread performance increases by about 30%.
Write performance enhancements vary greatly. The Write performance increase via compression should be split into three sets of data: When using “nodatacow” and “nodatasum”, the difference is minimal. Otherwise, when using any RAID, performance increases by about 40% (50 MB/s to 70 MB/s in most tests). Otherwise (”Standard” or “-m single” options), performance more than doubles (30 MB/s to 70 MB/s).
Read performance enhancements do not vary greatly, but are also certainly not 30% when using compression. There is a
April 22nd, 2009 12:16 PM (permalink)
Log in to Reply
laytonjb said: +0   
I’m not sure I follow you? Are you saying the read performance can vary by a great deal so 30% may be in the noise?
I didn’t run the tests a number of times and compute the average or examine the standard deviation in a systematic manner. But I did run them several times to get a feel for the results. The numbers I reported are right around the average but that is by eyeball :)
BTW - I’m not sure when they will declare Btrfs stable. I’m guessing early 2010. I talked to Chris but didn’t ask that question. They are still doing some fairly heavy lifting in the code base. I think the on-disk format is fixed but they could change it one more time (it’s still marked as “experimental” :) ).
Did I capture your questions correctly or am I missing something?
Thanks!
April 22nd, 2009 2:13 PM (permalink)
Log in to Reply
jimmershere said: +1   
Excellent article! I’m glad to see this kind of technology in the OpenSource world!
I do need to point out some incorrect entries in the BTRFS vs. ZFS comparison chart, however. specifically:
ZFS can online resize.
ZFS can repair/check/defrag while online. (scrub)
ZFS can take a snapshot of a snapshot
That being said, BTRFS still sounds like an exciting addition to Linux and I can’t wait to test it out.
April 22nd, 2009 1:14 PM (permalink)
Log in to Reply
laytonjb said: -1   
Thanks for the comments. Can you point to an on-line source for the ZFS features? I’ve been looking for some time for a comprehensive list of ZFS features without luck.
Thanks!
April 22nd, 2009 2:14 PM (permalink)
Log in to Reply
sysadmn said: +2   
One pretty good intro to ZFS is here:http://opensolaris.org/os/community/zfs/whatis/
It’s a glorified bullet list with a brief description of what each feature means.
For example, one could quibble on the “snapshot of snapshots” feature:
ZFS backup and restore are powered by snapshots. Any snapshot can generate a full backup, and any pair of snapshots can generate an incremental backup. Incremental backups are so efficient that they can be used for remote replication — e.g. to transmit an incremental update every 10 seconds.
It also appears you’ve left some of the ZFS advantages out of the table - I’d encourage your readers to see the original blog posting. I don’t know if this is in BTRFS, but it’s a lifesaver:
ZFS provides unlimited constant-time snapshots and clones. A snapshot is a read-only point-in-time copy of a filesystem, while a clone is a writable copy of a snapshot. Clones provide an extremely space-efficient way to store many copies of mostly-shared data such as workspaces, software installations, and diskless clients.
April 23rd, 2009 5:27 AM (permalink)
Log in to Reply
craig73 said: +1   
It would have been useful to have CPU usage statistics for all these tests to identify the cost of compression. I would have expected that the compression level/algorithm be selected to increase IO bandwidth at a slight cost to CPU overhead, but the lack of CPU numbers makes it difficult to evaluate how much of a cost that 30% came at.
[I’m also curious how tunable this is, based on larger CPUs and larger caches, or specific workloads]
April 22nd, 2009 1:27 PM (permalink)
Log in to Reply
laytonjb said: +0   
You’re absolutely correct but I didn’t track the CPU usage during the run :( Bonnie++ might be an easier way to get this information.
I have an interview with Chris Mason that will be appearing soon. He pointed out that the crc32 computations used in the compression can now be done in hardware with Nehalem. I hope to have a Nehalem system soon and I will definitely test this out.
If you want me to repost the testing and to add more tests, let the editor know :)
Thanks!
April 22nd, 2009 2:17 PM (permalink)
Log in to Reply
craig73 said: +3   
(BTW - I recognize the compression of zero’s makes the compression numbers meaningless)
April 22nd, 2009 1:28 PM (permalink)
Log in to Reply
Trackback:s5h.net » Blog Archive » Linux Doesn’t Need Oracle’s ZFS Because of Oracle’s Btrfs
Trackback:Btrfs the other upcoming FS « Alpha’s Place
Trackback:IRC: #boycottnovell @ FreeNode: April 22nd, 2009 - Part 2 | Boycott Novell
Trackback:IRC: #boycottnovell @ FreeNode: April 22nd, 2009 - Part 2 | All about MICROSOFT
Trackback:Twitted by wariola
Trackback:Twitted by pho_bos
mark_w said: +2   
I am very glad to see this comparison, but I would like to make a few comments.
- the title/summary is poor. The comment “Linux don’t need no stinkin’ ZFS…“Butter FS” is on the horizon and it’s boasting more features and better performance” is misleading.
According to your summary table, you have one area in which BTRFS can be considered more featured than ZFS (snapshots of snapshots). Considering that you haven’t even discussed clones (which maybe do the same thing or more) and that ZFS is not only a filesystem but it, and its utilities, integrate the LVM features and it has, right now, a wide selection of RAID modes that are actually functioning and it does ‘automagic’ adaptation to volumes of differing performance values, this seems to be a claim that you get nowhere near justifying. Maybe, eventually, it will be true, but the current version doesn’t have the features and there is little evidence that ‘first stable release’ will either.
(Sorry, I’m not trying to say that BTRFS is ‘bad’, just that you are accidentally overclaiming; my guess would be that you haven’t read much on ZFS and all of its features. I think if you had, you would have realised that BTRFS on its own or in its first stable release isn’t going to do it. BTRFS plus on the fly compression, plus a revised MD, plus LVM may well, though, but I’m expecting to wait a bit for all of those to work ideally together.)
- the title leads you to believe that BTRFS will be shown to have a better performance than ZFS. The actual comparison is between BTRFS and ext filesystems with various configurations.
- one of the clever features of ZFS is protection against silent data corruption. This may not be vital to me now, but as filesystem sizes increase…a colleague deals with the situation in which a research project grabs nearly 1G of data every day and they have data going back over 5 years (and they intend to keep capturing data for another 50+ years). As he says “how do I ensure that the data that we read is the data that we wrote?”, and its a good question. ZFS is good in that, admittedly obscure, situation, but I’m not sure whether BTRFS will be.
- You make (indirectly) the very good point that with the more advanced filesystems, benchmarking performance can be rather sensitive to set-up options. I am very glad that you did that testing and hope that you can do more thorough testing as we get closer to a version 1.0 release. I am not sure that I can help you with your funding submission, though…
It is also the case that the results that you get are heavily dependant on what you measure, so the more different tests you do, the better.
- another clever feature of ZFS (sorry about sounding like sounding like a stuck record, but I am impressed by ZFS and would love to be as impressed by BTRFS) is that you can make an array incorporating SSDs and hard disks and have the system automagically sort out the usage of the various storage types to give an agregate high performance, low-cost-per-TB, array. In a commercial setting this is probably one of the biggest advantages that ZFS has over earlier technology. I’m sure that I don’t have to use the ‘O’ word here, but this is a vital advantage that Sun has, and I’m expecting to see some interesting work on optimising this system, in particular for database accesses, soon.
(BTW, this is so effective that a hybrid array, as used in Amber Road, can be so effective that comparing a hybrid array of SSDs and slow hard disks can be faster, cheaper and much more power efficient than one using the usual enterprise class SAS disks, so this can be an interesting solution, in enterprise apps)
If you wish to know more about ZFS, you can do worse than read Sun’s whitepapers on the subject; search on “sun zfs discovery day” to get more info; the presentation is very readable. (I’m not, btw, suggesting that this is an unbiased comparison of ZFS to anything else; its a description of the features and how-to-use.)
April 23rd, 2009 2:38 PM (permalink)
Log in to Reply
smino said: +0   
I was going to say your trasnfer speeds are slow, as I get 70MB/s trasnfer files over ethernet Gige from XP to Unraid cache drive (csche meaning single drive unprotected no raid) which uses rieserFS. The drives are use are two 500GB Seagte 7200.12 and seagate 1.5TB 7200.11. I am sure the performance would be better trasnfering from drive to drive and not over ethernet. I am pretty close to hitting the Ethernet practical limit. Of course this is with my aqntivirus running and zone alarm pro, and streaming a movie from the same box.
If anyone gets a chance to test the brts performance on a fast system with those seagate drives above, please let me know.
I chose those drives because they are as fast at the veloraptor 10K drives, only more storage and cheaper!
Sly.
April 24th, 2009 6:42 AM (permalink)
Log in to Reply
Trackback:» Btrfs, linux non ha bisogno di Zfs
Trackback:Another day in development… » File Systems, Disk Defragmentation and more.
softweyr said: +0   
One (of the many) mis-understood features of ZFS is how the file metadata works. In most existing filesystems that support arbitrary metadata, the metadata space is limited and essentially allows key/data pairs to be associated with the file. In ZFS, the metadata for each filesystem object is another entire filesystem rooted at the containing file. ZFS is, in essence, a 3D filesystem. Consider Apple’s application bundles, currently implemented as a directory that the Finder application recognizes and displays specially. Using ZFS, the application file could be the icon file for the application, all of the component files, including executables, libraries, localization packs, and other app resources, would be stored in the file metadata. To the outside world, the bundle truly appears as a single file.
ZFS has it’s warts, for instance that storage of small files is quite inefficient, but actually besting it is going to be a long, hard slog. I suspect the big commercial Linux distributions will iron out the licensing issues somehow and including ZFS in their distributions in the next year. Linux is too big a business to be driven by the religious zealots anymore.
May 4th, 2009 6:51 AM (permalink)
Log in to Reply
Comments via RSS
Pleaselog in to post a comment.
Don't have an account?Register now for free access to all of LinuxMagazine.com
<a href="http://d1.openx.org/ck.php?oaparams=2__bannerid=74441__zoneid=11135__cb=8b389d6dad__oadest=http://ad.doubleclick.net/jump/N3271.linux-mag.com/B3613383;click0=http://d1.openx.org/ck.php?oaparams=2__bannerid=74441__zoneid=11135__cb=8b389d6dad__oadest=;abr=!ie4;abr=!ie5;sz=336x280;ord=8b389d6dad?" target="_blank"> <IMG SRC="http://ad.doubleclick.net/ad/N3271.linux-mag.com/B3613383;click0=http://d1.openx.org/ck.php?oaparams=2__bannerid=74441__zoneid=11135__cb=8b389d6dad__oadest=;abr=!ie4;abr=!ie5;sz=336x280;ord=8b389d6dad?" BORDER=0 WIDTH=336 HEIGHT=280 ALT="Click Here"></A>
Recent Recommendations [?]
The Other Desktop: Running Xfce
SSH Tunneling
Top 100 Linux Web Sites
Micro-Frameworks: Big Things in Small Packages
Trimming the FAT: Linux and Patents
Incremental Twiddling
Music on the Command-Line
VMware vSphere: Out of the Box and into the Clouds
Linus on Linux: The Linus Torvalds Interview Part 1
Harder, Better, Faster, Stronger: New Features in Postgres 8.4
Search Terms
iscsi
git
raid and lvm, part three
ftrace
clamav
compact flas reader xandros linux
compact flash
parallel platters part two
"data warehouse"
ssh tunnel
Tags [?]
linux
wizard boot camp
hpc
ldap
ubuntu
virtualization
perl
mpi 2
shells
samba
View all tags
Confronting the Data Center Crisis: A Cost-Benefit Analysis of the IBM Computing on Demand (CoD) Cloud Offering from IBM
Quantum StorNext - High-performance, Heterogeneous Shared File System from Quantum
NetApp and VMware Virtual Infrastructure 3 Storage Best Practices from NetApp
Virtualization Everywhere from Citrix
More »
Interview: Intel's Richard Dracott (Part One) Doug Eadline talks with Intel's Richard Dracott, General Manager of the High Performance Computing Organization.
IBM Blue Gene is the World's Fastest Supercomputer Doug Eadline visits the IBM booth at SC'07 to get a look at IBM's Blue Gene.

IBM System x iDataPlex: The Newest Economical Workhorse in the Computing Cloud for Next Generation HPC Data Centers
Georgia Tech Implements a Cool Solution for Green HPC with IBM
TGen Accelerates Genomic Research with IBM System Cluster 1350

Solving Power and Cooling Challenges for High-Performance Computing
Quad-Core Processors Bring Higher Performance and Lower Cost to Mainstream Computing
Intel Dual-Core HPC Cluster: A High-Performance Computing Testbed of Next-Generation Intel Xeon Processors
IBMIntel
<a href='http://d1.openx.org/ck.php?n=af69ea2e&amp;cb=INSERT_RANDOM_NUMBER_HERE' target='_blank'><img src='http://d1.openx.org/avw.php?zoneid=20190&amp;cb=INSERT_RANDOM_NUMBER_HERE&amp;n=af69ea2e' border='0' alt='' /></a> <a href='http://d1.openx.org/ck.php?n=a2a1267e&amp;cb=INSERT_RANDOM_NUMBER_HERE' target='_blank'><img src='http://d1.openx.org/avw.php?zoneid=20191&amp;n=a2a1267e' border='0' alt='' /></a>
<a href='http://d1.openx.org/ck.php?n=a702116b&amp;cb=INSERT_RANDOM_NUMBER_HERE' target='_blank'><img src='http://d1.openx.org/avw.php?zoneid=20192&amp;cb=INSERT_RANDOM_NUMBER_HERE&amp;n=a702116b' border='0' alt='' /></a> <a href='http://d1.openx.org/ck.php?n=a72204f5&amp;cb=INSERT_RANDOM_NUMBER_HERE' target='_blank'><img src='http://d1.openx.org/avw.php?zoneid=20193&amp;n=a72204f5' border='0' alt='' /></a>
Linux Magazine Job Board
Researcher - Cloud Computing
Beijing, China-PRC, China
Intel
Java Developer
Beijing, Beijing, China
USense
$50 Referral Reward
Software Engineer Intern
Shanghai, China-PRC, China
Action Script Programmer
China
Interactive Selection
System Intern
Shanghai, China-PRC, China
Post a Job >
Powered by JobThread

',1)">
ActivSupport
Linux Magazine has chosen ActivSupport asIT consultants.
Sponsored Links
Interop Ability from Microsoft and Novell.
See how business is running with it @moreinterop.com
HP Insight Dynamics-VSE software
Spin the IBM Wheel of I.T. Solutions.
Rackspace
LinuxMagazine.com is hosted by Rackspace

About  |  Contact Us  |  Privacy Policy  |  FAQ  |  RSS
© Linux Magazine 1999-2009 All rights reserved.
LinuxMagazine.com v4.0