Some questions/ideas about Fuse

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Some questions/ideas about Fuse

Csaba Henk
Hi!

During doing the FreeBSD port, I bumped into some quirks of Linux Fuse I
couldn't find out how they work.

Miklos, I'd like to ask you about these.

* Fuse specific inode data is kept in a structure called fuse_inode.
  What would be a natural idea is to insert this data to the appropriate
  slot of an inode structure (generic_ip, if I understand correctly).
 
  Fuse does it in the other way: normal inodes are kept within
  fuse_inodes (as far as I see, the inode allocation mechanism is gently
  bastardized to allocate memory accodingly).
 
  Of course, syscall handlers operate on normal inodes, so there must be a
  way to get the corresponding fuse_inode from a normal one. This is done
  by a fishy pointer arithmetic hack (the "container_of" macro). Why so?
  Why don't you use the above mentioned "natural" approach? Or calling
  that natural is just my personal preconception?
 
* Fuse nodeids are stored on 64 bits, while standard inode numbers live
  on 32. And these get casted back and forth. Don't you think it can
  cause some problem sometime somehow? Or that it's plain ugly? Why is it
  this way?

* I don't understand how readdir can sensibly operate with the off field
  of dirents (I speak of the case when a non-zero "off" argument is
  passed to readdir_fill). The problem is that what finally hits the
  kernel is the private fuse_dirent structure, opaque to the actual
  userspace filesystem (if it doesn't go beyond highlevel lib). How can
  that then realistically prescribe the layout of dirents within the
  directory stream? And for what purpose?
 
  The place where I could see dir offsets in use is the fusexmp_fh code.
  I just don't understand how can that work with simply passing on the
  real d_off field of Linux dirents (this value seems to be too small:
  fuse_dirents are bigger than normal ones). [Note that in FreeBSD there
  is no d_off field, rather a d_reclen thingy is used. To make fusexmp_fh
  work with FreeBSD, my first attempt was to sum the d_reclen values, and
  use the sum as offset. That didn't work, and that's what I'd expect in
  my current state of mind. Now I just use the joker 0 offset, and it
  works fine.]

* I neither understand how fuse_requests are managed. A simple messaging
  session consists of fuse_get_request, request_send, fuse_put_request.
  The first two of these push outstanding_sem down (or raise
  outstanding_debt), while fuse_put_reques pushes the semafor down (or
  decrease debt). As debt seems to be a way of extending the semafor below
  zero (ie., think of "semafor - debt" as the value of the extended
  semafor), it seems that the net gain of a simple messaging session is
  -1. Hence the prediction is that we quickly run out of available
  requests, and everything gets frozen. Reality is a bit more graceful
  than this though. At what point did I go wrong or what factlet did I
  skip?

And some ideas:

These feature ideas are brewed from the way using sshfs felt, but they
might be general enough to be added to the Fuse library.

* Uid (gid) mapping. Have an option which works similarly to "uid=n", but
  instead of replacing all uids with a constant value, this should
  happen only with one particular uid (so the option takes two uids, the
  "from" and the "to" uids). Or maybe you could specify a list of uids to
  be replaced with another one. Syntax? Maybe overloading uid like
  "uid=1003,1004:1005" ?

  Rationale: when I have uid 1001 at the local box and I sshfs to a
  remote one where my uid is 1003, seeing my files as belonging to an user
  of 1003 is uncomfortable; and using an allover uid=1001 yields
  information loss.

  You can argue that sanitizing one uid doesn't help the others which
  remain misleading, so it's just a bad hack. If so, maybe a "map this
  (list of) uid(s) to that uid, and map the rest to uid 1" semantics
  could be applied.

* Symlink mapping. If I have an ssh account where /home is a symlink to
  /usr/home, accessing that /home via sshfs will lead to unexpected
  results. Hence an option could be made which prefixes absolute
  symlinks with a given path (the mountpoint could be a practical
  default). Thus the remote /home won't appear to me as a symlink to
  /usr/home, but as a symlink to, eg., /mnt/ssh/usr/home, which is a
  valid remote path.

What do you think of these? Would you implement these in some form, or
accept a patch which does this?

Regards,
Csaba



-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Some questions/ideas about Fuse

Miklos Szeredi
> During doing the FreeBSD port,

Great work!

> I bumped into some quirks of Linux Fuse I couldn't find out how they
> work.
>
> Miklos, I'd like to ask you about these.
>
> * Fuse specific inode data is kept in a structure called fuse_inode.
>   What would be a natural idea is to insert this data to the appropriate
>   slot of an inode structure (generic_ip, if I understand correctly).
>  
>   Fuse does it in the other way: normal inodes are kept within
>   fuse_inodes (as far as I see, the inode allocation mechanism is gently
>   bastardized to allocate memory accodingly).
>  
>   Of course, syscall handlers operate on normal inodes, so there must be a
>   way to get the corresponding fuse_inode from a normal one. This is done
>   by a fishy pointer arithmetic hack (the "container_of" macro). Why so?
>   Why don't you use the above mentioned "natural" approach? Or calling
>   that natural is just my personal preconception?

There's no real difference between the two approaches.  There are
slight advantages to the current method: one less memory allocation
and one less pointer dereference for accessing the fuse_inode data.

Using container_of() is analogous to casting a base class pointer to a
derived class pointer in C++ terminology.  You can actually implement
multiple inheritance with container_of().  The list_head "class" is
very often used in such a multiple inheritance scenario.

> * Fuse nodeids are stored on 64 bits, while standard inode numbers live
>   on 32. And these get casted back and forth. Don't you think it can
>   cause some problem sometime somehow? Or that it's plain ugly? Why is it
>   this way?

Actually nodeids and inode numbers are completely separate on the
kernel interface and implementation.  The first is stored in
fuse_inode->nodeid, the second is stored in inode->i_ino.

So why do we need the two types of IDs?  Well, inode->i_ino is used
_only_ for filling in st_ino in the *stat() syscalls.  While nodeid is
used to identify the actual node as looked up by the userspace
filesystem.

Why is this good?  For example, on the path based API a nodeid
identifies a path.  But two paths may actually refer to the same inode
(hard link).  So the filesystem can use the 'use_ino' mount option and
fill in the st_ino field to the "real" inode number, while the library
may independently use the nodeid to perform lookup and construct
paths.

Yes, this is sort of a hack, but it seems to work well, and
conceptually quite clean.

> * I don't understand how readdir can sensibly operate with the off field
>   of dirents (I speak of the case when a non-zero "off" argument is
>   passed to readdir_fill). The problem is that what finally hits the
>   kernel is the private fuse_dirent structure, opaque to the actual
>   userspace filesystem (if it doesn't go beyond highlevel lib). How can
>   that then realistically prescribe the layout of dirents within the
>   directory stream? And for what purpose?

In linux 'd_off' does not necessarily have do with the layout of the
directory stream.  It's a cookie which represents the file offset
where the next entry is found.  It doesn't even have to be monothonic
(it should be unique though).

>   The place where I could see dir offsets in use is the fusexmp_fh code.
>   I just don't understand how can that work with simply passing on the
>   real d_off field of Linux dirents (this value seems to be too small:
>   fuse_dirents are bigger than normal ones). [Note that in FreeBSD there
>   is no d_off field,

Hmm, SuS doesn't mandate a d_off field, and doesn't specify it's
semantics, so fusexmp_fh is not portable this way.

>   rather a d_reclen thingy is used. To make fusexmp_fh
>   work with FreeBSD, my first attempt was to sum the d_reclen values, and
>   use the sum as offset. That didn't work, and that's what I'd expect in
>   my current state of mind. Now I just use the joker 0 offset, and it
>   works fine.]

That's OK.

Doing a telldir() after each readdir() and giving that as an offset
should also work.

> * I neither understand how fuse_requests are managed. A simple messaging
>   session consists of fuse_get_request, request_send, fuse_put_request.
>   The first two of these push outstanding_sem down (or raise
>   outstanding_debt), while fuse_put_reques pushes the semafor down (or
>   decrease debt). As debt seems to be a way of extending the semafor below
>   zero (ie., think of "semafor - debt" as the value of the extended
>   semafor), it seems that the net gain of a simple messaging session is
>   -1.

No, request_send() only downs the semaphore (or increases debt), if
it's not a preallocated request (i.e. not one obtained from
fuse_get_request()).

> And some ideas:
>
> These feature ideas are brewed from the way using sshfs felt, but they
> might be general enough to be added to the Fuse library.
>
> * Uid (gid) mapping. Have an option which works similarly to "uid=n", but
>   instead of replacing all uids with a constant value, this should
>   happen only with one particular uid (so the option takes two uids, the
>   "from" and the "to" uids). Or maybe you could specify a list of uids to
>   be replaced with another one. Syntax? Maybe overloading uid like
>   "uid=1003,1004:1005" ?

Sounds good.  For However it would be better to add it as a separate
layer.  It's quite easy to stack functionality like this (see the
cache layer in sshfs).

Such general purpose layers could be distributed with the library,
maybe even with an automatic stacker which would add layers specified
in mount options.

> * Symlink mapping. If I have an ssh account where /home is a symlink to
>   /usr/home, accessing that /home via sshfs will lead to unexpected
>   results. Hence an option could be made which prefixes absolute
>   symlinks with a given path (the mountpoint could be a practical
>   default). Thus the remote /home won't appear to me as a symlink to
>   /usr/home, but as a symlink to, eg., /mnt/ssh/usr/home, which is a
>   valid remote path.

Absolute symlinks are evil.  This is an often requested feature and
I'm still trying to resist.

Maybe a layer which does a conversion to relative symlinks (which the
sysadmin should have done) could be acceptable.

> What do you think of these? Would you implement these in some form, or
> accept a patch which does this?

I don't promise to apply patches, but at least I'll consider doing so ;)

Thanks,
Miklos


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Some questions/ideas about Fuse

Miklos Szeredi
> Using container_of() is analogous to casting a base class pointer to a
> derived class pointer in C++ terminology.  You can actually implement
> multiple inheritance with container_of().  The list_head "class" is
> very often used in such a multiple inheritance scenario.

In fact using multiple list_heads within a single object couldn't be
done with C++ inheritance, since there's no way to inherit multiple
times from the same base class.

I've always known that C++ sucks ;)

Miklos


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Some questions/ideas about Fuse

Csaba Henk
In reply to this post by Miklos Szeredi
On 2005-09-22, Miklos Szeredi <[hidden email]> wrote:
>> During doing the FreeBSD port,
>
> Great work!

Thanks. I try...

>> * Fuse specific inode data is kept in a structure called fuse_inode.
>>   What would be a natural idea is to insert this data to the appropriate
>>   slot of an inode structure (generic_ip, if I understand correctly).
>>  
>>   Fuse does it in the other way: normal inodes are kept within
>>   fuse_inodes (as far as I see, the inode allocation mechanism is gently
>>   bastardized to allocate memory accodingly).
>>  
>>   Of course, syscall handlers operate on normal inodes, so there must be a
>>   way to get the corresponding fuse_inode from a normal one. This is done
>>   by a fishy pointer arithmetic hack (the "container_of" macro). Why so?
>>   Why don't you use the above mentioned "natural" approach? Or calling
>>   that natural is just my personal preconception?
>
> There's no real difference between the two approaches.  There are
> slight advantages to the current method: one less memory allocation
> and one less pointer dereference for accessing the fuse_inode data.
>
> Using container_of() is analogous to casting a base class pointer to a
> derived class pointer in C++ terminology.  You can actually implement
> multiple inheritance with container_of().  The list_head "class" is
> very often used in such a multiple inheritance scenario.

OK, I see.

>> * Fuse nodeids are stored on 64 bits, while standard inode numbers live
>>   on 32. And these get casted back and forth. Don't you think it can
>>   cause some problem sometime somehow? Or that it's plain ugly? Why is it
>>   this way?
>
> Actually nodeids and inode numbers are completely separate on the
> kernel interface and implementation.  The first is stored in
> fuse_inode->nodeid, the second is stored in inode->i_ino.
>
> So why do we need the two types of IDs?  Well, inode->i_ino is used
> _only_ for filling in st_ino in the *stat() syscalls.  While nodeid is
> used to identify the actual node as looked up by the userspace
> filesystem.

Maybe not "_only_"?

When you set FUSE_READDIR_INO, the dirent's d_ino (Linux) / d_fileno
(BSD) field is filled with a value casted from the nodeid (and this
field is stored on 32 bits in FreeBSD, and on 32 or 64 bits on Linux
[depending on the architecture]).

The funny thing here is that FUSE_READDIR_INO is always set in FreeBSD,
as getcwd(3) relies on d_fileno when it tries to identify pathname
components (contrary to looking them up). See: an OS functionality which
relies on casted nodeids... [Shells and some other beasts tend to use
their own getcwd-alike [[lookup based ?]] which is not sensitive to
d_fileno, but there are a bunch of programs which are confused by a
non-working getcwd.]

> Why is this good?  For example, on the path based API a nodeid
> identifies a path.  But two paths may actually refer to the same inode
> (hard link).  So the filesystem can use the 'use_ino' mount option and
> fill in the st_ino field to the "real" inode number, while the library
> may independently use the nodeid to perform lookup and construct
> paths.
>
> Yes, this is sort of a hack, but it seems to work well, and
> conceptually quite clean.

Hm. I forcibly disabled FUSE_USE_INO under FreeBSD as it blocks
FUSE_READDIR_INO... although as I think into it, it smells like as if those
filesystems which can make use of FUSE_USE_INO will also be able to get
getcwd right with FUSE_USE_INO being set. So maybe I should allow
FUSE_USE_INO...

>> * I don't understand how readdir can sensibly operate with the off field
>> of dirents (I speak of the case when a non-zero "off" argument is passed
>> to readdir_fill). The problem is that what finally hits the kernel is the
>> private fuse_dirent structure, opaque to the actual userspace filesystem
>> (if it doesn't go beyond highlevel lib). How can that then realistically
>> prescribe the layout of dirents within the directory stream? And for what
>> purpose?
>
> In linux 'd_off' does not necessarily have do with the layout of the
> directory stream.  It's a cookie which represents the file offset where the
> next entry is found.  It doesn't even have to be monothonic (it should be
> unique though).

Heh... doing funky arithmetics in kernel with fuse_dirents' "off" (as I
did) is tad bogus, so you say?

>>   rather a d_reclen thingy is used. To make fusexmp_fh
>>   work with FreeBSD, my first attempt was to sum the d_reclen values, and
>>   use the sum as offset. That didn't work, and that's what I'd expect in
>>   my current state of mind. Now I just use the joker 0 offset, and it
>>   works fine.]
>
> That's OK.
>
> Doing a telldir() after each readdir() and giving that as an offset
> should also work.

Note that in FreeBSD telldir is not as cheap as in Linux where simply the
appropriate field of a dirent is returned -- eg., in FreeBSD it might involve
locking. (Cf.

http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/gen/telldir.c

) Not as if optimizaiton should be an issue in an example filesystem...

>> * I neither understand how fuse_requests are managed. A simple messaging
>>   session consists of fuse_get_request, request_send, fuse_put_request.
>>   The first two of these push outstanding_sem down (or raise
>>   outstanding_debt), while fuse_put_reques pushes the semafor down (or
>>   decrease debt). As debt seems to be a way of extending the semafor below
>>   zero (ie., think of "semafor - debt" as the value of the extended
>>   semafor), it seems that the net gain of a simple messaging session is
>>   -1.
>
> No, request_send() only downs the semaphore (or increases debt), if
> it's not a preallocated request (i.e. not one obtained from
> fuse_get_request()).

Uhh, I overlooked the bang before "req->preallocated" in queue_request...

>> * Uid (gid) mapping. Have an option which works similarly to "uid=n", but
[...]
>
> Sounds good.  For However it would be better to add it as a separate
> layer.  It's quite easy to stack functionality like this (see the
> cache layer in sshfs).
>
> Such general purpose layers could be distributed with the library,
> maybe even with an automatic stacker which would add layers specified
> in mount options.

Does it mean that you wouldn't wanna deal with such features until such a
frameworklet is added to the lib, or just means that you'd prefer a stacking
kinda approach over overloading existing options?

>> * Symlink mapping. If I have an ssh account where /home is a symlink to
[...]
> Absolute symlinks are evil.  This is an often requested feature and
> I'm still trying to resist.
>
> Maybe a layer which does a conversion to relative symlinks (which the
> sysadmin should have done) could be acceptable.

What do you mean by "sysadmin should have done"? AFAIK Fuse "supports"
malicious daemons in the sense that the system can't be broken by
them... and there is not much more it could do in this respect (the
possible benefits of setuid bits end when the daemon puts her hands on
the file descriptor to the device). Fuse also supports well behaved
daemons in the sense that behaving well is made easy for them. That's
fine and dandy. Would you say the story doesn't end here?

Or do you imply that the conversion should take place in the kernel?
Then you won't get a patch for it :)

> I don't promise to apply patches, but at least I'll consider doing so ;)

OK :)

Csaba



-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Re: Some questions/ideas about Fuse

Miklos Szeredi
> > So why do we need the two types of IDs?  Well, inode->i_ino is used
> > _only_ for filling in st_ino in the *stat() syscalls.  While nodeid is
> > used to identify the actual node as looked up by the userspace
> > filesystem.
>
> Maybe not "_only_"?
>
> When you set FUSE_READDIR_INO, the dirent's d_ino (Linux) / d_fileno
> (BSD) field is filled with a value casted from the nodeid (and this
> field is stored on 32 bits in FreeBSD, and on 32 or 64 bits on Linux
> [depending on the architecture]).

Yes, but this is a userspace issue.  The library does reuse the nodeid
as the inode number (st_ino) if 'use_ino' mount option is not present.
And as you say it's also used to fill in 'd_ino' if 'readdir_ino' is
present.

So 'use_ino' with 'readdir_ino' try to make 'd_ino' to be in sync with
'st_ino', but it cannot make guess d_ino, unless the node has already
been looked up: i.e. first directory read will return dummy d_ino
values.

If 'use_ino' is not present, it's up to the filesystem to supply
consistent 'st_ino' and 'd_ino' values.

> The funny thing here is that FUSE_READDIR_INO is always set in FreeBSD,
> as getcwd(3) relies on d_fileno when it tries to identify pathname
> components (contrary to looking them up). See: an OS functionality which
> relies on casted nodeids... [Shells and some other beasts tend to use
> their own getcwd-alike [[lookup based ?]] which is not sensitive to
> d_fileno, but there are a bunch of programs which are confused by a
> non-working getcwd.]

OK, so does the 64/32 bit conversion present some problems?  Do you
use 'nodeid' or 'ino' as d_fileno?

> > Why is this good?  For example, on the path based API a nodeid
> > identifies a path.  But two paths may actually refer to the same inode
> > (hard link).  So the filesystem can use the 'use_ino' mount option and
> > fill in the st_ino field to the "real" inode number, while the library
> > may independently use the nodeid to perform lookup and construct
> > paths.
> >
> > Yes, this is sort of a hack, but it seems to work well, and
> > conceptually quite clean.
>
> Hm. I forcibly disabled FUSE_USE_INO under FreeBSD as it blocks
> FUSE_READDIR_INO... although as I think into it, it smells like as if those
> filesystems which can make use of FUSE_USE_INO will also be able to get
> getcwd right with FUSE_USE_INO being set. So maybe I should allow
> FUSE_USE_INO...

Yes.

> >> * I don't understand how readdir can sensibly operate with the off field
> >> of dirents (I speak of the case when a non-zero "off" argument is passed
> >> to readdir_fill). The problem is that what finally hits the kernel is the
> >> private fuse_dirent structure, opaque to the actual userspace filesystem
> >> (if it doesn't go beyond highlevel lib). How can that then realistically
> >> prescribe the layout of dirents within the directory stream? And for what
> >> purpose?
> >
> > In linux 'd_off' does not necessarily have do with the layout of the
> > directory stream.  It's a cookie which represents the file offset where the
> > next entry is found.  It doesn't even have to be monothonic (it should be
> > unique though).
>
> Heh... doing funky arithmetics in kernel with fuse_dirents' "off" (as I
> did) is tad bogus, so you say?

Quite.  If *BSD doesn't have a dirent->d_off field, I doubt, that you
can make use of the offset field, other than reading through the
directory always supplying the offset in the last entry as the new
offset.  IOW you cannot do seeks in the directory stream.

> >>   rather a d_reclen thingy is used. To make fusexmp_fh
> >>   work with FreeBSD, my first attempt was to sum the d_reclen values, and
> >>   use the sum as offset. That didn't work, and that's what I'd expect in
> >>   my current state of mind. Now I just use the joker 0 offset, and it
> >>   works fine.]
> >
> > That's OK.
> >
> > Doing a telldir() after each readdir() and giving that as an offset
> > should also work.
>
> Note that in FreeBSD telldir is not as cheap as in Linux where simply the
> appropriate field of a dirent is returned -- eg., in FreeBSD it might involve
> locking. (Cf.
>
> http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/gen/telldir.c
>
> ) Not as if optimizaiton should be an issue in an example filesystem...

Ahh, and it also involves storing the position in a linear list, and
searching through that list in seekdir().  Not very efficient to do a
telldir() on each entry then.

It also explains how directory stream seeking is done without d_off.

In this case, I think you've done the best thing by setting the 'off'
parameter to zero, since that offset would not be used by anything and
would just slow down the diretory reading.

> > Sounds good.  For However it would be better to add it as a separate
> > layer.  It's quite easy to stack functionality like this (see the
> > cache layer in sshfs).
> >
> > Such general purpose layers could be distributed with the library,
> > maybe even with an automatic stacker which would add layers specified
> > in mount options.
>
> Does it mean that you wouldn't wanna deal with such features until such a
> frameworklet is added to the lib, or just means that you'd prefer a stacking
> kinda approach over overloading existing options?

I prefer stacking instead of adding too much cruft to basic framework.
So while the 'uid=num' option is very simple to implement, a more
compex uid/gid mapping belongs in a 'layer'.

> >> * Symlink mapping. If I have an ssh account where /home is a symlink to
> [...]
> > Absolute symlinks are evil.  This is an often requested feature and
> > I'm still trying to resist.
> >
> > Maybe a layer which does a conversion to relative symlinks (which the
> > sysadmin should have done) could be acceptable.
>
> What do you mean by "sysadmin should have done"? AFAIK Fuse "supports"
> malicious daemons in the sense that the system can't be broken by
> them...

Yes.  I didn't mean, that absolute symlinks are a security problem,
rather that they cause problems when exported or mounted at different
a different place.  There's usually _no_ justification to use absolute
symlinks, and it's mosty a case of just ingorance or lazyness on the
part of the person creating the symlink.

> and there is not much more it could do in this respect (the
> possible benefits of setuid bits end when the daemon puts her hands on
> the file descriptor to the device). Fuse also supports well behaved
> daemons in the sense that behaving well is made easy for them. That's
> fine and dandy. Would you say the story doesn't end here?
>
> Or do you imply that the conversion should take place in the kernel?
> Then you won't get a patch for it :)

No, I mean, that either you replace all absolute symlinks by hand (or
by script) with relative ones, or (in case you are not the sysadmin,
and can't do the above) you write a layer that does this on the fly.

Miklos


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Re: Some questions/ideas about Fuse

Anton Altaparmakov
On Sun, 25 Sep 2005, Miklos Szeredi wrote:

> > >> * Symlink mapping. If I have an ssh account where /home is a symlink to
> > [...]
> > > Absolute symlinks are evil.  This is an often requested feature and
> > > I'm still trying to resist.
> > >
> > > Maybe a layer which does a conversion to relative symlinks (which the
> > > sysadmin should have done) could be acceptable.
> >
> > What do you mean by "sysadmin should have done"? AFAIK Fuse "supports"
> > malicious daemons in the sense that the system can't be broken by
> > them...
>
> Yes.  I didn't mean, that absolute symlinks are a security problem,
> rather that they cause problems when exported or mounted at different
> a different place.  There's usually _no_ justification to use absolute
> symlinks, and it's mosty a case of just ingorance or lazyness on the
> part of the person creating the symlink.
>
> > and there is not much more it could do in this respect (the
> > possible benefits of setuid bits end when the daemon puts her hands on
> > the file descriptor to the device). Fuse also supports well behaved
> > daemons in the sense that behaving well is made easy for them. That's
> > fine and dandy. Would you say the story doesn't end here?
> >
> > Or do you imply that the conversion should take place in the kernel?
> > Then you won't get a patch for it :)
>
> No, I mean, that either you replace all absolute symlinks by hand (or
> by script) with relative ones, or (in case you are not the sysadmin,
> and can't do the above) you write a layer that does this on the fly.

You may like relative symlinks but there are many cases where relative
symlinks are evil and absolute ones are a must.  For example on all my
systems I only use absolute symlinks because many directories are
symlinked and relative symlinks then cause breakage, e.g.

/usr/src is a symlink to /mnt/home/src.

$ cd /usr/src/linux-2.6
$ cd ../..
$ pwd
/mnt/home

So a symlink inside /usr/src that points out of /usr/src in a relative
manner would suddenly find itself in /mnt/home instead of in /usr and
would no longer resolve...  An absolute one going to /usr/... always
works.

Just my 2p.

Best regards,

        Anton
--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Re: Some questions/ideas about Fuse

Miklos Szeredi
> You may like relative symlinks but there are many cases where relative
> symlinks are evil and absolute ones are a must.  For example on all my
> systems I only use absolute symlinks because many directories are
> symlinked and relative symlinks then cause breakage, e.g.
>
> /usr/src is a symlink to /mnt/home/src.
>
> $ cd /usr/src/linux-2.6
> $ cd ../..
> $ pwd
> /mnt/home
>
> So a symlink inside /usr/src that points out of /usr/src in a relative
> manner would suddenly find itself in /mnt/home instead of in /usr and
> would no longer resolve...  An absolute one going to /usr/... always
> works.

The breakage you describe happens when you _change_ /usr/src to be a
link.

Yes, relative symlinks are more fragile when moving things around.
But if things don't move (which they should rarely do), they are
superior.

Miklos



-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Re: Some questions/ideas about Fuse

Anton Altaparmakov
On Sun, 25 Sep 2005, Miklos Szeredi wrote:

> > You may like relative symlinks but there are many cases where relative
> > symlinks are evil and absolute ones are a must.  For example on all my
> > systems I only use absolute symlinks because many directories are
> > symlinked and relative symlinks then cause breakage, e.g.
> >
> > /usr/src is a symlink to /mnt/home/src.
> >
> > $ cd /usr/src/linux-2.6
> > $ cd ../..
> > $ pwd
> > /mnt/home
> >
> > So a symlink inside /usr/src that points out of /usr/src in a relative
> > manner would suddenly find itself in /mnt/home instead of in /usr and
> > would no longer resolve...  An absolute one going to /usr/... always
> > works.
>
> The breakage you describe happens when you _change_ /usr/src to be a
> link.

No it doesn't.  It is permanent.  I always have /usr/src/ a symlink.  It
was just a random example.

> Yes, relative symlinks are more fragile when moving things around.
> But if things don't move (which they should rarely do), they are
> superior.

Moving is irrelevant.  My point is that relative symlinks have a huge
element of surprise because going back through them (what a relative
symlink always has to do - it will always start with a number of ../../...
in it) will get to a different directory that one expects.  If you have
problems with /usr/src/, I will give you a different example:

/usr/local/bin is a symlink to /mnt/local/bin.

I as sysadmin want to create a symlink in /usr/local/bin called vim that
will point to /usr/bin/vim.  So logically I can do two things:

cd /usr/local/bin
ln -s /usr/bin/vim vim

This will work fine.

Or I can do:

cd /usr/local/bin
ln -s ../../bin/vim vim

But that creates an invalid symlink because the ../.. gets us to
/mnt instead of /usr so there is no bin/vim in there.

See?  It completely is against the element of least surprise.

And anyway, e.g. Solaris does not allow relative symlinks for a lot of
things (e.g. the mount command will only accept full paths, not relative
ones, as will lofiadm) and even on Linux you can only use absolute paths
for some things (e.g. some git commands which use rsync only work with an
absolute path).  So for portability relative symlinks are very bad, too.

Best regards,

        Anton
--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Re: Some questions/ideas about Fuse

Miklos Szeredi
> No it doesn't.  It is permanent.  I always have /usr/src/ a symlink.  It
> was just a random example.
>
> > Yes, relative symlinks are more fragile when moving things around.
> > But if things don't move (which they should rarely do), they are
> > superior.
>
> Moving is irrelevant.  My point is that relative symlinks have a huge
> element of surprise because going back through them (what a relative
> symlink always has to do - it will always start with a number of ../../...
> in it) will get to a different directory that one expects.  If you have
> problems with /usr/src/, I will give you a different example:
>
> /usr/local/bin is a symlink to /mnt/local/bin.
>
> I as sysadmin want to create a symlink in /usr/local/bin called vim that
> will point to /usr/bin/vim.  So logically I can do two things:
>
> cd /usr/local/bin
> ln -s /usr/bin/vim vim
>
> This will work fine.
>
> Or I can do:
>
> cd /usr/local/bin
> ln -s ../../bin/vim vim
>
> But that creates an invalid symlink because the ../.. gets us to
> /mnt instead of /usr so there is no bin/vim in there.

Ahh, but you should know, that /usr/local/bin is at /mnt/local/bin and
do the linking accordingly:

ln -s ../../../usr/bin/vim .

> See?  It completely is against the element of least surprise.

With relative links you have to know the real location where you are
linking from.  That's why they are fragile on moving.

But absolute symlinks are fragile on changing the mountpoint of the
root (as the case for NFS/sshfs/etc exporting).

More generally absolute symlinks break if you move the root.  Relative
symlinks break if you move something that is contained in the link.

> And anyway, e.g. Solaris does not allow relative symlinks for a lot of
> things (e.g. the mount command will only accept full paths, not relative
> ones, as will lofiadm) and even on Linux you can only use absolute paths
> for some things (e.g. some git commands which use rsync only work with an
> absolute path).  So for portability relative symlinks are very bad, too.

I don't know about these.  I generally found, that working with
relative symlinks cause much less problems than absolute ones.

Miklos


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Re: Some questions/ideas about Fuse

Anton Altaparmakov
On Sun, 25 Sep 2005, Miklos Szeredi wrote:

> > No it doesn't.  It is permanent.  I always have /usr/src/ a symlink.  It
> > was just a random example.
> >
> > > Yes, relative symlinks are more fragile when moving things around.
> > > But if things don't move (which they should rarely do), they are
> > > superior.
> >
> > Moving is irrelevant.  My point is that relative symlinks have a huge
> > element of surprise because going back through them (what a relative
> > symlink always has to do - it will always start with a number of ../../...
> > in it) will get to a different directory that one expects.  If you have
> > problems with /usr/src/, I will give you a different example:
> >
> > /usr/local/bin is a symlink to /mnt/local/bin.
> >
> > I as sysadmin want to create a symlink in /usr/local/bin called vim that
> > will point to /usr/bin/vim.  So logically I can do two things:
> >
> > cd /usr/local/bin
> > ln -s /usr/bin/vim vim
> >
> > This will work fine.
> >
> > Or I can do:
> >
> > cd /usr/local/bin
> > ln -s ../../bin/vim vim
> >
> > But that creates an invalid symlink because the ../.. gets us to
> > /mnt instead of /usr so there is no bin/vim in there.
>
> Ahh, but you should know, that /usr/local/bin is at /mnt/local/bin and
> do the linking accordingly:
>
> ln -s ../../../usr/bin/vim .

Indeed but if your average user does this on a multi user system (just in
his home directory somewhere) he gets caught out by it by surprise.

> > See?  It completely is against the element of least surprise.
>
> With relative links you have to know the real location where you are
> linking from.  That's why they are fragile on moving.
>
> But absolute symlinks are fragile on changing the mountpoint of the
> root (as the case for NFS/sshfs/etc exporting).
>
> More generally absolute symlinks break if you move the root.  Relative
> symlinks break if you move something that is contained in the link.
>
> > And anyway, e.g. Solaris does not allow relative symlinks for a lot of
> > things (e.g. the mount command will only accept full paths, not relative
> > ones, as will lofiadm) and even on Linux you can only use absolute paths
> > for some things (e.g. some git commands which use rsync only work with an
> > absolute path).  So for portability relative symlinks are very bad, too.
>
> I don't know about these.  I generally found, that working with
> relative symlinks cause much less problems than absolute ones.

That is fine.  I was just saying that some people might have experienced
the opposite.  I am one of those people.  The only reason I chimed in was
to point out that relative symlinks are not the answer to everything,
that's all.

Best regards,

        Anton
--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Some questions/ideas about Fuse

Csaba Henk
In reply to this post by Miklos Szeredi
On 2005-09-25, Miklos Szeredi <[hidden email]> wrote:

>> When you set FUSE_READDIR_INO, the dirent's d_ino (Linux) / d_fileno
>> (BSD) field is filled with a value casted from the nodeid (and this
>> field is stored on 32 bits in FreeBSD, and on 32 or 64 bits on Linux
>> [depending on the architecture]).
>
> Yes, but this is a userspace issue.  The library does reuse the nodeid
> as the inode number (st_ino) if 'use_ino' mount option is not present.
> And as you say it's also used to fill in 'd_ino' if 'readdir_ino' is
> present.
>
> So 'use_ino' with 'readdir_ino' try to make 'd_ino' to be in sync with
> 'st_ino', but it cannot make guess d_ino, unless the node has already
> been looked up: i.e. first directory read will return dummy d_ino
> values.

If you do a getcwd, elements of the path to be found out are already
looked up, so this is not a problem.

> If 'use_ino' is not present, it's up to the filesystem to supply
> consistent 'st_ino' and 'd_ino' values.
>
>> The funny thing here is that FUSE_READDIR_INO is always set in FreeBSD,
>> as getcwd(3) relies on d_fileno when it tries to identify pathname
>> components (contrary to looking them up). See: an OS functionality which
>> relies on casted nodeids... [Shells and some other beasts tend to use
>> their own getcwd-alike [[lookup based ?]] which is not sensitive to
>> d_fileno, but there are a bunch of programs which are confused by a
>> non-working getcwd.]
>
> OK, so does the 64/32 bit conversion present some problems?  Do you
> use 'nodeid' or 'ino' as d_fileno?

As FreeBSD's getcwd relies on st_ino and d_fileno, a collision of
d_fileno-s (of dirents pulled from a certain dir) could lead to
malfunction. When I do a readdir I use fuse_dirents' 64 bit ino to fill
normal dirents' 32 bit d_fileno (what else ?). The origin of
fuse_dirents' ino is up to userspace but a 64/32 conversion takes place
for sure.

> In this case, I think you've done the best thing by setting the 'off'
> parameter to zero, since that offset would not be used by anything and
> would just slow down the diretory reading.

Noo... setting off to zero is ostrich politics :)

As I can't exclude bumping into a filesystem which uses 'off', I aim to
make my code be able to work with it.

>> >> * Symlink mapping. If I have an ssh account where /home is a symlink to
>> [...]
>> > Absolute symlinks are evil.  This is an often requested feature and
>> > I'm still trying to resist.
>> >
>> > Maybe a layer which does a conversion to relative symlinks (which the
>> > sysadmin should have done) could be acceptable.
>>
>> What do you mean by "sysadmin should have done"? AFAIK Fuse "supports"
>> malicious daemons in the sense that the system can't be broken by
>> them...
>
> Yes.  I didn't mean, that absolute symlinks are a security problem,
> rather that they cause problems when exported or mounted at different
> a different place.  There's usually _no_ justification to use absolute
> symlinks, and it's mosty a case of just ingorance or lazyness on the
> part of the person creating the symlink.

[
 ... Here was my two paragraph opinion on absolute/relative symlinks.
  *Erased*
]

But it's not my real point. My real point is that my point is irrelevant
(see erasure above). Yours, too. We can feel about abs/rel symlinks as
we feel like, nevertheless they keep on existing.

Eg., in FreeBSD /home is a symlink to /usr/home. Whether it's a problem
for an sshfs user is not determined by her/his own OS but the ssh
server's OS (so it's not my personal pain, but might be yours, too).

> No, I mean, that either you replace all absolute symlinks by hand (or
> by script) with relative ones, or (in case you are not the sysadmin,
> and can't do the above) you write a layer that does this on the fly.

OK, I got it. With sshfs it should be considered typical that the user
is not an admin on the remote host.

Considering a layer in the lib which relativizes symlinks on the fly...
as I think into it, it's a feature I don't want.

Both absolute and relative symlinks has their own semantics, which might
be prone to misuse and confusing, but at least, it's well defined and
predictable.

Relativized absolute symlinks will look like relative symlinks but
behave like absolute (eg., the path they refer to will be resistent to
moving [as long as you keep it in the filesystem]). Now *that* is evil.

I think sanitizing those crappy absolute symlinks would be a useful
feature. But if you do it so that you hurt them in their absoluteness
it will just add to the chaos.

Csaba



-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel