Fuse + socket = fuse/nfs?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Fuse + socket = fuse/nfs?

Edwin Olson
Has anyone done the obvious thing and slapped a TCP socket between the
libfuse interface and a fuse "pass-through" file system, so that you can
mount a remote file system? i.e., send all the fuse fn calls over the
socket. This could be a compelling alternative to NFS, but providing an
opportunity to make a couple simple improvements (secure authentication
and transport encryption come to mind!)

Would this work?

What sorts of performance issues would occur?

Would adding flock to the fuse API largely address locking concerns, or
is more mechanism needed?

-Ed



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fuse + socket = fuse/nfs?

Miklos Szeredi
> Has anyone done the obvious thing and slapped a TCP socket between the
> libfuse interface and a fuse "pass-through" file system, so that you can
> mount a remote file system? i.e., send all the fuse fn calls over the
> socket. This could be a compelling alternative to NFS, but providing an
> opportunity to make a couple simple improvements (secure authentication
> and transport encryption come to mind!)
>
> Would this work?

Well, if the server and client have the same endiannes, then yes.
Otherwise you'd have some trouble with swapping byte order on every
field.

> What sorts of performance issues would occur?
>
> Would adding flock to the fuse API largely address locking concerns, or
> is more mechanism needed?

I'm not sure it's possible to do proper locking in userspace.  Does
the userspace NFS server support file locking?

Miklos


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fuse + socket = fuse/nfs?

Martin C. Atkins
In reply to this post by Edwin Olson
On Sat, 18 Jun 2005 14:14:42 -0400 Edwin Olson <[hidden email]> wrote:
> Has anyone done the obvious thing and slapped a TCP socket between the
> libfuse interface and a fuse "pass-through" file system, so that you can
> mount a remote file system? i.e., send all the fuse fn calls over the
> socket. This could be a compelling alternative to NFS, but providing an
> opportunity to make a couple simple improvements (secure authentication
> and transport encryption come to mind!)

This is pretty much what the v9fs driver that was recently added to
the -mm tree does, except that the protocol is already endianness,
architecture, and (largely) operating system independent (it already
talks to three different operating systems: Linux, Plan 9 and
Inferno).

The advantages over NFS are much as you state. I would add that it also
makes remote synthetic file systems easy (and this would also be true of fuse).

The obvious question is: why do we need two such protocols, when one
would do? Well OSS is all about choice! (and, of course, there
are always going to be some areas where each protocol will do better)

Martin
PS
 Linux Weekly News had a nice introduction to v9fs a couple of weeks ago,
it is towards the end of:
        http://lwn.net/Articles/136579/
for anyone interested...
--
Martin C. Atkins [hidden email]
Parvat Infotech Private Limited http://www.parvat.com{/,/martin}


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fuse + socket = fuse/nfs?

Martin PLACEK
In reply to this post by Edwin Olson
> Message: 4
> To: [hidden email]
> CC: [hidden email]
> Subject: Re: [fuse-devel] Fuse + socket = fuse/nfs?
> From: Miklos Szeredi <[hidden email]>
> Date: Sat, 18 Jun 2005 23:14:25 +0200
>
> > Has anyone done the obvious thing and slapped a TCP socket between the
> > libfuse interface and a fuse "pass-through" file system,

   yes, its work in progress for me! I'm doing it as part of my
Masters by Research. The status atm is it supports as you say
a "pass-through" file system, but I'd like it do a bit more than that,
stuff like replication and aggregation of storage, plus more...
I'm treating the project as a learning process at the moment, the code
isn't too bad, but I'd like to get it to state that I'm happy with
before considering releasing it out just yet.

> > so that you can
> > mount a remote file system? i.e., send all the fuse fn calls over the
> > socket. This could be a compelling alternative to NFS, but providing an
> > opportunity to make a couple simple improvements (secure authentication
> > and transport encryption come to mind!)
> >
> > Would this work?
>
> Well, if the server and client have the same endiannes, then yes.
> Otherwise you'd have some trouble with swapping byte order on every
> field.

   very true, atm, I'm assuming same endiannes, and I'd imagine structs like
statfs would change even between platforms aswell eg SunOS vs Linux vs...  even
differing version of the same OS they would change.  (hint: look at the C
source for 'df' command) so you can't just pass those structs around either. So
for the moment I'll have to assume no only same endianess but same version of
OS. I wonder how the file i/o structs will change in nature with 64bit
platforms?


>
> > What sorts of performance issues would occur?

   With no caching, which is what I've got at the moment the
performance is so.. so.. . Its just gets bad when you do a `ls -l` in a
dir of hundreds of files as it has to do a GETATTR for every single file and
it all happens synchronously so that is a bit painfull, caching that sort
of stuff will help heaps I'd imagine. I found increasing the block
size to help tremendously with perfomance, network latency is a killer when
your shifting around many small blocks synchronously....

> >
> > Would adding flock to the fuse API largely address locking concerns, or
> > is more mechanism needed?
>
> I'm not sure it's possible to do proper locking in userspace.  Does
> the userspace NFS server support file locking?

   I'm not too sure what to do about consistency and how it would work with fuse,
for the moment the plan would be to do the locking on the server end. Start of
at file level (lock on opens/unlock on close) than possibly attempt to do it at
block level.. But I'm not really an expert here so if anyone can give me any
pointers here I'd really appreciate it!


Oh and before I forget:

Thanks Miklos for writing such a great piece of software.. all the best!


>
> Miklos
>
>
> --__--__--
>


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fuse + socket = fuse/nfs?

Edwin Olson
In reply to this post by Miklos Szeredi

>Well, if the server and client have the same endiannes, then yes.
>Otherwise you'd have some trouble with swapping byte order on every
>field.
>
>  
>
hton and ntoh are par for the course :)

>I'm not sure it's possible to do proper locking in userspace.  Does
>the userspace NFS server support file locking?
>
>  
>
I am not an expert, but I see two major (potential) locking problems.
For the moment, let's ignore performance and just consider correctness.

1. Explicit locking support, i.e., implementing flock/lockf. I can't
forsee any difficulty in implementing this in userland. I know very
little about linux VFS specifics, but if libfuse passed those calls to
me, I believe I could do the right thing on my end. AFAIK, all that NFS
lockd does is implement this level of locking.

2. VFS consistency. Even if the fusefs and applications are cooperating
on locking, it does no good if the VFS layer is caching metadata or data
without regard for the network file system's locking and consistency
requirements. Consistency can be broken down into metadata and data:

    Metadata: From a previous discussion (5/22/05), Miklos Szeredi wrote
that metadata (dentrys) can be programmed to 'expire' after a preset
amount of time. Currently this is set to 1 second, which provides a
reasonable compromise between performance and consistency for most
applications, but so long as this time delay is known and bounded, the
file system can guarantee consistency by holding on to locks longer than
ideally required (i.e., wait for the dentry to expire before releasing
the lock.)

    Data: We'd want similar expiration of cached data too. Miklos wrote
on 6/18/06 that -odirect_io can be used to "make each read() start a new
request", which sounds like a solution to this problem as well.

In summary, I think that locking is do-able from userspace through a
combination of -odirect_io, setting short dentry expirations, and
implementing flock/lockf. Would anyone care to
correct/comment/elaborate? Did I miss anything?

-Ed

PS: Miklos- feel free to tear me a new one if I've misrepresented your
emails!



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fuse + socket = fuse/nfs?

Miklos Szeredi
> I am not an expert, but I see two major (potential) locking problems.
> For the moment, let's ignore performance and just consider correctness.
>
> 1. Explicit locking support, i.e., implementing flock/lockf. I can't
> forsee any difficulty in implementing this in userland. I know very
> little about linux VFS specifics, but if libfuse passed those calls to
> me, I believe I could do the right thing on my end. AFAIK, all that NFS
> lockd does is implement this level of locking.

It's a bit more complicated than this.  See this thread:

http://thread.gmane.org/gmane.comp.file-systems.fuse.devel/598

> 2. VFS consistency. Even if the fusefs and applications are cooperating
> on locking, it does no good if the VFS layer is caching metadata or data
> without regard for the network file system's locking and consistency
> requirements.

There's a simple solution to this: flush data/metadata caches for the
inode on lock/unlock operations.  Other than this consistency is
usually not a requirement for network filesystems.

Miklos


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Loading...