NILFS - A New Implementation of a Log-structured File System for Linux
========================================================================

What is NILFS?
--------------

NILFS is a new implementation of a log-structured file system for the
Linux 2.6 operating system.  Conventional features of a log-structured
file systems (LFS) are high write performance and fast recovery
time.  In addition, LFS prevents data write operations from overwriting
the disk blocks, thereby minimizing the damage to file data and system
consistency on hardware failure.  Furthermore, LFS writes data and
meta-data in the correct order which helps ensuring consistency in the
entire file system.  LFS can instantaneously create file system
snapshots and check the file system speedy.  We implemented ``NILFS''
using modern technology: For example, the file and inode blocks are
managed by a B-tree structure, internal data are processed in 64 bit
wide word size.  The B-tree structure enables ``NILFS'' to create and
store huge files.  

What is NILFS version 2?
------------------------

NILFS version 2 (NILFS2) is a new version of NILFS redesigned for
pactical use.  NILFS2 realizes online garbage collection that reclaims
disk space with keeping multiple snapshots.  As well as preserving
recent data and namespaces of a filesystem which save users from
unexpected data lost, NILFS2 can keep selected filesystem checkpoints
(i.e. snapshots) for long periods, for an indefinite period if you
want.  This flexible snapshot feature helps you to maintain
significant versions of data or system state, and would make
snapshotting or versioning of the POSIX filesystem much familiar to
you.  Snapshot administration is easy and quickly performable as well
as its creation is instantaneous and spontaneous like NILFS version 1.
You can receive full benefits of snapshotting without wasting valuable
time.

The possible use of NILFS2 includes disaster recovery, versioning,
tamper detection, SOX compliance logging, and so on.  It can serve as
an alternative filesystem for Linux desktop environment, or as a basis
of advanced storage appliances.

License
-------

NILFS2 is a freely-distributable open source software, and is
released under the GNU General Public License (GPLv2).

Prerequisites
-------------

Make sure you have installed all of the followings:

- Linux 2.6.18~2.6.29
  The kernel should be used on the build machine, and it should be
  compiled from the source.  In some distros, kernel header packages
  are available instead.  For example, Debian users can skip a
  kernel build for distro kernels by installing the corresponding
  linux-headers-2.6-xxx package.

  Though it is not guaranteed to work on distro kernels,
  some patches are available on the following FAQ page:

  http://nilfs.sourceforge.net/en/faq.html

- GCC
- GNU Make
- GNU Binutils
- Development package of uuid library (e.g., e2fsprogs-devel or
  uuid-dev)
- newer umount utility

If you use official or unofficial distro binary packages described
later, these are omissible.

NILFS2 was tested on some i386-architecture machines, x86_64 and
ppc64 machines.

Installation
------------

1. For distro kernels, install official or unofficial binary packages
   of NILFS2 module and NILFS2 utilities.
   Or get source tar ball files for vanilla kernels.

   The unofficial binary packages and the source tar balls are 
   available on http://nilfs.sourceforge.net/en/download.html.

2. Compile and install NILFS2 (skip this if using the binary packages)

  Kernel module
   # tar jxf nilfs-2.x.x.tar.bz2
   # cd nilfs-2.x.x
   # make
   # make install
   (you should run "make uninstall" if you have already installed)

   The file "/lib/modules/`uname -r`/kernel/nilfs/nilfs2.ko" will be
   created.

  Userland tools
   # tar jxf nilfs-utils-2.x.x.tar.bz2
   # cd nilfs-utils-2.x.x
   # ./configure
   # make
   # make install

3. Format a disk partition and mount it

   e.g.
   # mkfs -t nilfs2 /dev/sdb1
   mkfs.nilfs2 ver 2.0
   Start writing file system initial data to the device
          Blocksize:4096  Device:/dev/sdb1  Device Size:73402366464
   File system initialization succeeded !! 
   # mkdir /nilfs
   # mount -t nilfs2 /dev/sdb1 /nilfs

   This will invoke a cleaner (i.e. garbage collector) through an
   external mount program (mount.nilfs2).  The cleaner is
   implemented as an userland daemon whose name is ``nilfs_cleanerd''.

4. How to make a snapshot
   For NILFS2, we distinguish two terminologies, ``checkpoint'' and
   ``snapshot'':

   A checkpoint represents a consistent state of NILFS2 filesystem.
   A number of checkpoints are created automatically and continuously.
   Though the recent checkpoints are protected from the cleaner
   during the period given by a cleaner parameter ``protection_period'',
   older checkpoints may be deleted by the cleaner.

   You can protect the older checkpoint by extending the
   protection_period with a larger value.
   For example, if you want to ensure to keep files made yesterday,
   specify larger (> 86400) protection_period in
   /etc/nilfs_cleanerd.conf.

   You can also do this by changing a checkpoint to a ``snapshot''.
   A snapshot is the checkpoint marked not to be deleted by the
   cleaner.  The checkpoint and the snapshot are handled through the
   following userland tools:

   lscp     lists checkpoints and snapshots.
   mkcp     makes a checkpoint immediately.
   mkcp -s  makes a snapshot of the current state immediately.
   chcp     changes an existing checkpoint to a snapshot or vice versa.
   rmcp     invalidates specified checkpoint(s).

   In the following example, the existing checkpoint whose checkpoint
   number is two, is changed into a snapshot after a period of time.

   # mount -t nilfs2 /dev/sdb1 /nilfs
   # --- some operation ---
   # lscp
                 CNO        DATE     TIME  MODE  SKT   NBLKINC       ICNT
                   1  2008-05-08 14:45:49  cp     -         11          3
                   2  2008-05-08 14:50:22  cp     -     200523         81
                   3  2008-05-08 20:40:34  cp     -        136         61
                   4  2008-05-08 20:41:20  cp     -     187666       1604
                   5  2008-05-08 20:41:42  cp     -         51       1634
                   6  2008-05-08 20:42:00  cp     -         37       1653
                   7  2008-05-08 20:42:42  cp     -     272146       2116
                   8  2008-05-08 20:43:13  cp     -     264649       2117
                   9  2008-05-08 20:43:44  cp     -     285848       2117
                  10  2008-05-08 20:44:16  cp     -     139876       7357
   # chcp ss 2
   # lscp
                 CNO        DATE     TIME  MODE  SKT   NBLKINC       ICNT
                   1  2008-05-08 14:45:49  cp     -         11          3
                   2  2008-05-08 14:50:22  ss     -     200523         81
                   3  2008-05-08 20:40:34  cp     -        136         61
                   4  2008-05-08 20:41:20  cp     -     187666       1604
                   5  2008-05-08 20:41:42  cp     -         51       1634
                   6  2008-05-08 20:42:00  cp     -         37       1653
                   7  2008-05-08 20:42:42  cp     -     272146       2116
                   8  2008-05-08 20:43:13  cp     -     264649       2117
                   9  2008-05-08 20:43:44  cp     -     285848       2117
                  10  2008-05-08 20:44:16  cp     -     139876       7357
                  11  2008-05-08 21:05:23  cp     -         10       7357

   Each checkpoint or snapshot is identified with a unique number
   called checkpoint number (CNO).  There is no practical limit on the 
   number of checkpoints and snapshots.

5. How to mount snapshots

   Snapshots are mountable as read-only filesystems.  They are mounted
   with two options, a read-only option (``-r'' or ``-o ro'') and the
   ``cp'' option which specifies the checkpoint number:

   # mount -t nilfs2 -r -o cp=2 /dev/sdb1 /nilfs-cp
   # df -t nilfs2
   Filesystem           1K-blocks      Used Available Use% Mounted on
   /dev/sdb1             71679996   3203068  64888832   5% /nilfs
   /dev/sdb1             71679996   3203068  64888832   5% /nilfs-cp
   # mount -t nilfs2
   /dev/sdb1 on /nilfs type nilfs2 (rw,gcpid=13296)
   /dev/sdb1 on /nilfs-cp type nilfs2 (ro,cp=2)

   The ``current'' filesystem and snapshots are mountable
   independently, which means that the online backup is possible
   with NILFS.

6. Snapshot operations

lscp - list checkpoints
lscp [-rsh] [device]
    -r  list in reverse order
    -s  list snapshots only
    -h  display help message and exit
    ----
    List checkpoints and snapshots of specified NILFS2 filesystem.

   # lscp
                 CNO        DATE     TIME  MODE  SKT   NBLKINC       ICNT
                   1  2008-05-08 14:45:49  cp     -         11          3
                   2  2008-05-08 14:50:22  ss     -     200523         81
                   3  2008-05-08 20:40:34  cp     -        136         61
                   4  2008-05-08 20:41:20  cp     -     187666       1604
		        checkpoint/snapshot^            ^number of blocks
  checkpoint number^  ^creation time                    number of inodes^     

mkcp - make a checkpoint
mkcp [-sh] [device]
    -s  create a snapshot
    -h  display help message and exit
    ----
    Flush in-core buffer data to disk, then make a checkpoint
    that holds current state of the file system.  You can make a snapshot
    with -s option.

chcp - change checkpoint mode
chcp [-h] ss/cp [device] checkpoint_number...
     -h  display help message and exit
     ss  make snapshot
     cp  revert snapshot to checkpoint
    ----
    Change checkpoint mode, checkpoints to snapshots and vise versa.
    Multiple checkpoint numbers are accepted.

rmcp - remove checkpoints
rmcp [-fih] [device] checkpoint_number...
    -f  force 
    -i  interactive
    ----
    Remove checkpoints forever, never be back.

Good luck!

-- The NILFS Development Team, NTT Laboratories

Copyright (C) 2007, 2008 Nippon Telegraph and Telephone Corporation.