Overview

In this sequence of labs, you'll build a multi-server file system called Yet-Another File System (yfs) in the spirit of Frangipani. At the end of all the labs, your file server architecture will look like this:

You'll write a file server process, labeled yfs above, using the FUSE toolkit. Each client host will run a copy of yfs. yfs will appear to local applications on the same machine by registering via FUSE to receive file system events from the operating system. The yfs extent server will store all the file system data on an extent server on the network, instead of on a local disk. yfs servers on multiple client hosts can share the file system by sharing a single extent server.

This architecture is appealing because (in principle) it shouldn't slow down very much as you add client hosts. Most of the complexity is in the per-client yfs program, so new clients make use of their own CPUs rather than competing with existing clients for the server's CPU. The extent server is shared, but hopefully it's simple and fast enough to handle a large number of clients. In contrast, a conventional NFS server is pretty complex (it has a complete file system implementation) so it's more likely to be a bottleneck when shared by many NFS clients.

Lab assignments

Lab 1 - Lock Server
Lab 2 - Basic File Server
Lab 3 - File Server: Reading, Writing and Sharing Files
Lab 4 - MKDIR, REMOVE, and Locking
Lab 5 - Caching locks
Lab 6 - Caching Extent Server + Consistency
Lab 7 - Paxos
Lab 8 - Replicated lock server
Lab 9 - Final Project

Collaboration Policy

You must write all the code you hand in for the programming assignments, except for code that we give you as part of the assignment. You are not allowed to look at anyone else's solution (and you're not allowed to look at code from previous years). You may discuss the assignments with other students, but you may not look at or copy each others' code.

Acknowledgements

This lab series have been adapted from those originally developed for the MIT class 6.824 by Robert Morris, Frans Kaashoek and the 6.824 staff.

Programming Environment

You need to do all labs on a Linux machine with pthread and FUSE support. In principle, any UNIX-style machine such as FreeBSD or MacOS would work, however, there are minor annoying differences between FUSE on Linux and FUSE on other operating systems that may cause your code to fail our tests when it seems to pass for you. Thus, if you choose to do your labs on your own machine, please ensure that your assignment passes the tests on the class machines we have provided, because that's where we are going to grade the labs.

The class machines are called {brain,eye,heart}@news.cs.nyu.edu.

If you want to do labs on your own Linux machine, install FUSE by doing apt-get install libfuse2 libfuse-dev fuse-utils (if it's a Ubuntu/Debian box). Or, you can compile and install fuse-2.6-3 from source.

New: Linux VMWare Image with FUSE/Git/SSH

The MIT 6.824 staff have created a VMWare Linux image based on the Debian 4.0r0 server image with FUSE from Thought Police. We have augmented it with ssh and git. You can download it here:

ds-class-debian.zip (590 MB)
Non-root account: username=notroot, password=6.82fork
Root account: password=6.82fork

You should be able to run this with VMWare Player, which is available for free from VMWare. We have tested it on Linux; it should work for Windows as well. Note that the official environment for the labs is still the class machines, and that's where we will be testing your code. So you should always test your code on the class machine before handing it in! For example, it's likely that you need to implement both the CREATE and MKNOD operations as different operating systems send either CREATE or MKNOD in response to the creation of a file. Also note that we don't really have the energy or expertise to debug any VMWare problems you might have. This is just meant to be helpful for those of you who don't have personal Linux machines, and find it inconvenient to log in at a class machine.

There has been at least one report that running the code in VMWare is much slower than running it directly on hardware, and as a result, RPCs timeout more often even when RPC_LOSSY is unset. Be on the lookout for errors related to this.

NOTE:You may find that the network doesn't work when you boot the image up for the first time. This is a known problem with the Thought Police image. To fix this, run the following command as root: 

$ rm /etc/udev/rules.d/z25_persistent-net.rules && reboot

To install additional packages on this image, login as root and do apt-get NAME_OF_DEBIAN_PACKAGE.

Lab Programming Tips

pthreads

This series of labs is built around the POSIX threads (pthreads) programming model. The pthreads package allows you to run different tasks concurrently within a process, lock access to shared data during critical sections, and communicate between threads using shared data. A comprehensive guide to programming with pthreads can be found here: http://www.llnl.gov/computing/tutorials/pthreads/.

Pthread interfaces are somewhat cumbersome to use. You can (optionally) use our custom wrapper objects in pwrapper.h. Note that the skeleton code we provide you in the labs do not use these wrapper objects. It's up to you to change the code to use them.

The pwrapper.h provide two classes: pMutex and pCond that take care of pthread initializations for mutexes and conditional variables automatically. Their usage is quite straightforward. The class pScopedMutex provides a scoped lock by locking the pMutex argument in its constructor function and unlocks the corresponding pMutex in its destructor. Therefore, an easy way to perform subsystem/protocol locking in a function is to declare a pScopedMutex with the subsystem lock pMutex in the function. When the function returns, the compiler ensures that the pScopedMutex's destructor will be called, thus automatically unlocking the subsystem mutex. (It frees you from the burden of having to explicitly unlock the mutex in multiple places when there are multiple return points in a function.)

The following is an example function using scoped lock. 

void my_func() {
  pScopedMutex ps(&pmu); //pmu is a pMutex
  if (foo > 0)
    return 1;
  else
    return 0;
}
The following is an example function that does the same thing without using scoped lock. 
void my_func() {
  pthread_mutex_lock(&mu); //mu is a pthread_mutex_t that's already initialized.
  if (foo > 0) {
     pthread_mutex_unlock(&mu);
     return 1;
  }else{
    pthread_mutex_unlock(&mu);
    return 0;
  }
}

RPC

The labs use our own customized RPC system (instead of the standardized SUN RPC system). Our RPC system launches a new thread on the server for each RPC received; the thread is destroyed upon return from the RPC call. Our RPC system is quite simple. In particular, you need to provide your own RPC procedure numbers and marshalling routines for user-defined arguments and return types. You will find examples for doing this in the skeleton code we have provided you. More importantly, there is no type checking on RPC procedures across the rpc client and server. If the arguments specified in your RPC invocation at the client do not match the actual procedure's implementation at the server, you will see an assertion failure somewhere in the rpc.{cc,h} files during runtime.

C++ Standard Template Library (STL)

We recommend you use the Standard Template Library, a collection of C++ classes implementing many common data structures and algorithms. You will find the classes std::string, std::map and std::vector particularly useful during these labs.

Debugging

printf statements are always your friend when debugging any kind of problem in your programs. However, when programming in C/C++, you should always be familiar with gdb, the GNU debugger. You may find this gdb reference useful. Below introduces a few gdb tips for complete newbies:

If your program is crashing (segmentation fault), type gdb program core where program is the name of the binary executable to examine the core file. If you don't find the core file anywhere, type ulimited -c unlimted befor starting your program again. Once inside gdb, type bt to examine the stack trace when the segmentation fault happened.

While your programming is running, you can attach gdb to it by typing gdb program 1234. Again, program is the name of the binary executable. 1234 is the process number of your running program. Of course, you can choose to run your program with gdb from the beginning. If so, simply type gdb program. Then at the gdb prompt, type run.

While in gdb, you can set breakpoints (use gdb command b) to stop the execution at specific points, examine variable contents (use gdb command p), etc.

To apply a given gdb command to all threads in your program, prepend thread apply all to your command. For example, thread apply all bt dumps the backtrace for all threads.

UNIX network programming:

W. Richard Stevens' books ``UNIX Network Programming'' Volume 1 and 2 are classic references for network programming. If you are struggling with the sockets interface it could be a helpful purchase. See the suggested books list for other helpful references.