Distributed Storage Systems

Introduction

You'll start with the code you wrote for Lab 2, and add support for reading and writing file contents. You'll have to choose a representation with which to store file contents in the block server and implement the SETATTR, WRITE, and READ RPCs.

Continuing with the CCFS Server

brain$ cp /home/jinyang/lab3.tgz .
brain$ tar xzvf lab3.tgz

brain$ rsync -av lab2/ lab3/

Lab 3 builds upon Lab 2 so make sure you pass the Lab 2 tester before proceeding. When you're done with Lab 3, you should be able to read and write files in your file system. For example, start up the block server on one of the class machines:

hand$ cd ~/lab3
hand$ ./blockdbd 3772 &

brain$ ./ccfs dir hand 3772 &
root file handle: 2d1b68f779135270
brain$ cd /classfs/dir
brain$ echo hello > a
brain$ ls -lt
total 0
-rw-r--r--  0 root  wheel  6 Feb 16 13:52 a
brain$ cat a
hello
brain$ 

Your Lab 3 file server must support sharing the file system on other hosts via the block server. So when you're done with Lab 3 you should be able to do this on a third machine (e.g. heart):

heart$ ./ccfs dir hand 3772 2d1b68f779135270 &
[1] 6608
root file handle: 2d1b68f779135270 
heart$ cd /classfs/dir
heart$ ls -lt
total 0
-rw-r--r--  0 root  wheel  6 Feb 16 13:52 a
heart$ cat a
hello
heart$ 

Your Job

For writes, you should ignore the stable argument and always write the file to stable storage (the block server). You should wait for the put of the file's contents and attributes to complete before sending a reply to the client. This behavior is equivalent to performing writes with the a->stable argument set to FILE_SYNC.

If your server passes the tester (see below), then you are done. If you have questions about whether you have to implement specific pieces of NFS server functionality, then you should be guided by the tester: if you can pass the tests without implementing something, then don't bother implementing it.

Please don't modify the block server or its client interface (blockdbc.C and blockdbc.h), since we may test your file server against our own copy of the block server.

Testing

hand$ ~/lab3/blockdbd 3883 &

heart$ ./ccfs dir1 hand 3883 &
[1] 72422
root file handle: 2a868a18dad0f84e
heart$ ./ccfs dir2 hand 3883 2a868a18dad0f84e &
[3] 72595
root file handle: 2a868a18dad0f84e
heart$

Now you can use test-lab3.pl to test your file system by passing the two root directories that you just created:

heart$ ./test-lab3.pl /classfs/dir1 /classfs/dir2
Write and read one file: OK
Write and read a second file: OK
Overwrite an existing file: OK
Append to an existing file: OK
Write into the middle of an existing file: OK
Check that one cannot open non-existant file: OK
Check directory listing: OK
Read files via second server: OK
Check directory listing on second server: OK
Passed all tests

If test-lab3.pl exits without printing "Passed all tests!", then it thinks something is wrong with your file server.

You should re-start ccfs before you run the tester. If you run the tester more than once without re-starting ccfs, the tester is likely to see stale information cached by the NFS client for the second file system. You don't need to solve this problem for Lab 3, but if you're interested, the solution is to implement mtime and ctime for files and directories.

Hints

SETATTR

Here's how to extract the argument for the SETATTR RPC:

setattr3args *a = nc->Xtmpl getarg<setattr3args> ();
//nfs_fh3 a->object : the file handle that the client wants to set attributes
//sattr3 a->new_attributes : the new attrbutes
//sattrguard3 a->guard : ignore this argument 

fattr3 attr;
if (a->new_attributes.size.set)
   attr.size = *(a->new_attributes.size.val);
if (a->new_attributes.mtime.set == SET_TO_CLIENT_TIME) 
   attr.mtime = *(a->new_attributes.mtime.time);

wccstat3 *res = nc->Xtmpl getres<wccstat3> ();
res->set_status (NFS3_OK);
*res->wcc = make_wcc (old_attr, new_attr);
nc->reply (nc->getvoidres ());

WRITE

write3args *a = nc->Xtmpl getarg<write3args> ();
// nfs_fh3 a->file : the file handle where the client wants to write.
// uint64 a->offset : the offset to start the write
// uint32 a->count : the length of data to write
// stable_how a->stable : ignore
// opaque a->data : handle for the data

str newdata (a->data.base (), a->count);

After a WRITE the file length should be MAX(old_length, offset+count). A WRITE should never make a file shorter. If an application wants to overwrite a file, which can cause the file to be shorter, the NFS client will first send a SETATTR and then a WRITE. The SETATTR has new_attributes.size.set=true, and new_attributes.size.val=0, which should cause your server to set the file length to zero. The WRITE then writes the number of bytes specified by the client. The resulting file can have a shorter or longer length than the original file.

Note that you'll have to update the size of the file (in its fattr3) as a side effect of most WRITE RPCs and when SETATTR explicitly asks your server to do so.

READ

Here's how to generate a reply for the WRITE RPC:

write3res *res = nc->Xtmpl getres<write3res> ();
res->set_status(NFS3_OK);
res->resok->file_wcc = make_wcc (old_attr, new_attr);
res->resok->count = /* The number of bytes of data written */;
res->resok->committed = FILE_SYNC; //Since this is how we implemented the write.
nc->reply (nc->getvoidres ());

read3args *a = nc->Xtmpl getarg<read3args> ();
// nfs_fh3 a->file :  the file handle where the client wants to read.
// uint64 a->offset : the offset to start the read.
// uint32 a->count : the length of data to write.

char value[bytes_read];

read3res *res = nc->Xtmpl getres<read3res> ();
res->set_status (NFS3_OK);
res->resok->eof = /* true, if already end of file; otherwise, false. */;
res->resok->file_attributes.set_present (false);
res->resok->count = bytes_read;
res->resok->data.set (value, bytes_read);
nc->reply (nc->getvoidres ());

Collaboration policy

Handin procedure

% tar czf lab-3-handin.tgz *.[TCchx] Makefile
% chmod og= lab-3-handin.tgz