FUSE protocol documentation

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

FUSE protocol documentation

Manlio Perillo
Hi.

I plan to write a client for the FUSE kernel module; the programming
language will be Go.
There is already a good implementation https://github.com/hanwen/go-fuse
however since this is a personal project, I would like to write a
client from scratch.

Unfortunately there is no good documentation for the protocol, and the
wiki page:
https://github.com/libfuse/libfuse/wiki/Protocol-Sketch
is a bit out of date (several links are broken).

My plan is:

1) Fix all the broken URLs in the wiki page.
    Some urls points to the old libfuse project page, and some other
points to dead sites but are fortunately available on
https://archive.org/web/
2) Document the protocol in an easy to write text format
3) Make the document easy to parse by a machine, so that a simple code
generator can be written
    (like XCB, for XWindow server)
4) Submit a pull request for the inclusion of the documentation in
libfuse/doc/protocol.txt


Any objections (expecially for 1)?


Thanks
Manlio Perillo

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Antonio SJ Musumeci
Please! 

I've wanted to spend time doing the same but had gotten busy. My usecase for FUSE would benefit from a different userland library design (relative to libfuse) but needing to first half reverse engineer the protocol has kept me from exploring it.

If I can help in any way let me know.

On Wed, Apr 13, 2016 at 11:32 AM, Manlio Perillo <[hidden email]> wrote:
Hi.

I plan to write a client for the FUSE kernel module; the programming
language will be Go.
There is already a good implementation https://github.com/hanwen/go-fuse
however since this is a personal project, I would like to write a
client from scratch.

Unfortunately there is no good documentation for the protocol, and the
wiki page:
https://github.com/libfuse/libfuse/wiki/Protocol-Sketch
is a bit out of date (several links are broken).

My plan is:

1) Fix all the broken URLs in the wiki page.
    Some urls points to the old libfuse project page, and some other
points to dead sites but are fortunately available on
https://archive.org/web/
2) Document the protocol in an easy to write text format
3) Make the document easy to parse by a machine, so that a simple code
generator can be written
    (like XCB, for XWindow server)
4) Submit a pull request for the inclusion of the documentation in
libfuse/doc/protocol.txt


Any objections (expecially for 1)?


Thanks
Manlio Perillo

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Nikolaus Rath
In reply to this post by Manlio Perillo
On Apr 13 2016, Manlio Perillo <[hidden email]> wrote:
> There is already a good implementation https://github.com/hanwen/go-fuse
> however since this is a personal project, I would like to write a
> client from scratch.

This sounds like a very odd rationale for me. What's special about code
written for a personal project that makes it unsuitable for your
purpose?

> Unfortunately there is no good documentation for the protocol, and the
> wiki page:
> https://github.com/libfuse/libfuse/wiki/Protocol-Sketch
> is a bit out of date (several links are broken).
>
> My plan is:
>
> 1) Fix all the broken URLs in the wiki page.
>     Some urls points to the old libfuse project page, and some other
> points to dead sites but are fortunately available on
> https://archive.org/web/
> 2) Document the protocol in an easy to write text format
> 3) Make the document easy to parse by a machine, so that a simple code
> generator can be written
>     (like XCB, for XWindow server)
> 4) Submit a pull request for the inclusion of the documentation in
> libfuse/doc/protocol.txt
>
> Any objections (expecially for 1)?

Please go ahead! That will be very valuable no matter why you do it :-).



Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Manlio Perillo
On Wed, Apr 13, 2016 at 6:57 PM, Nikolaus Rath <[hidden email]> wrote:
> On Apr 13 2016, Manlio Perillo <[hidden email]> wrote:
>> There is already a good implementation https://github.com/hanwen/go-fuse
>> however since this is a personal project, I would like to write a
>> client from scratch.
>
> This sounds like a very odd rationale for me. What's special about code
> written for a personal project that makes it unsuitable for your
> purpose?
>

The special thing is that there are no deadlines for a personal
project, so I can take all the time I want.
For a normal project, you usually start with an already existing
library, and only modify/rewrite it if really necessary.

>> Unfortunately there is no good documentation for the protocol, and the
>> wiki page:
>> https://github.com/libfuse/libfuse/wiki/Protocol-Sketch
>> is a bit out of date (several links are broken).
>>
>> My plan is:
>>
>> 1) Fix all the broken URLs in the wiki page.
> [...]
>> Any objections (expecially for 1)?
>
> Please go ahead! That will be very valuable no matter why you do it :-).
>

I have updated the broken links on the wiki.

Links from from http://opengrok.creo.hu/fuselib/ and
http://fuse.cvs.sourceforge.net/fuse/ have been replaced by links to
https://github.com/libfuse/libfuse/blob/fuse_2_9_5/.
I have used a fixed tag to make links stable.

The link http://www.night.dataphone.se/~jrydberg/fuse.html has been
replaced with https://web.archive.org/web/20070503133705/http://www.night.dataphone.se/~jrydberg/fuse.html

Two remaining broken links are
http://opengrok.creo.hu/fuse4bsd/xref/fuse_module/fuse_msg.c#fuse_body_audit
and
http://fuse4bsd.creo.hu/darcsweb/darcsweb.cgi?r=fuse4bsd;a=headblob;f=/tools/fuse-structs.yml

since I was unable to find the source control repository of fuse4bsd.


Manlio

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Michael Zimmermann
> The special thing is that there are no deadlines for a personal
> project, so I can take all the time I want.
> For a normal project, you usually start with an already existing
> library, and only modify/rewrite it if really necessary.
This still doesn't make sense unless the whole point of your project is learning. Otherwise you'd have to write your own operating system and compiler first, too. :P

On Wed, Apr 13, 2016 at 8:08 PM, Manlio Perillo <[hidden email]> wrote:
On Wed, Apr 13, 2016 at 6:57 PM, Nikolaus Rath <[hidden email]> wrote:
> On Apr 13 2016, Manlio Perillo <[hidden email]> wrote:
>> There is already a good implementation https://github.com/hanwen/go-fuse
>> however since this is a personal project, I would like to write a
>> client from scratch.
>
> This sounds like a very odd rationale for me. What's special about code
> written for a personal project that makes it unsuitable for your
> purpose?
>

The special thing is that there are no deadlines for a personal
project, so I can take all the time I want.
For a normal project, you usually start with an already existing
library, and only modify/rewrite it if really necessary.

>> Unfortunately there is no good documentation for the protocol, and the
>> wiki page:
>> https://github.com/libfuse/libfuse/wiki/Protocol-Sketch
>> is a bit out of date (several links are broken).
>>
>> My plan is:
>>
>> 1) Fix all the broken URLs in the wiki page.
> [...]
>> Any objections (expecially for 1)?
>
> Please go ahead! That will be very valuable no matter why you do it :-).
>

I have updated the broken links on the wiki.

Links from from http://opengrok.creo.hu/fuselib/ and
http://fuse.cvs.sourceforge.net/fuse/ have been replaced by links to
https://github.com/libfuse/libfuse/blob/fuse_2_9_5/.
I have used a fixed tag to make links stable.

The link http://www.night.dataphone.se/~jrydberg/fuse.html has been
replaced with https://web.archive.org/web/20070503133705/http://www.night.dataphone.se/~jrydberg/fuse.html

Two remaining broken links are
http://opengrok.creo.hu/fuse4bsd/xref/fuse_module/fuse_msg.c#fuse_body_audit
and
http://fuse4bsd.creo.hu/darcsweb/darcsweb.cgi?r=fuse4bsd;a=headblob;f=/tools/fuse-structs.yml

since I was unable to find the source control repository of fuse4bsd.


Manlio

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Alejandro Exojo
On 14 April 2016 at 09:32, Michael Zimmermann <[hidden email]> wrote:
>
> > The special thing is that there are no deadlines for a personal
> > project, so I can take all the time I want.
> > For a normal project, you usually start with an already existing
> > library, and only modify/rewrite it if really necessary.
> This still doesn't make sense unless the whole point of your project is learning. Otherwise you'd have to write your own operating system and compiler first, too. :P

I disagree. For people to write file systems in other languages, a
protocol might be more helpful than a C library. Also, when all the
operations in your file system are asynchronous, the synchronous
interface of fuse_operations might be a problem.

I've already written my file system with the fuse library, and is OK,
but I would be very thankful to have the option to speak a protocol
instead.

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Nikolaus Rath
On Apr 14 2016, Alejandro Exojo <alex-efHHZNW+34gk9YCHp/[hidden email]> wrote:

> On 14 April 2016 at 09:32, Michael Zimmermann <[hidden email]> wrote:
>>
>> > The special thing is that there are no deadlines for a personal
>> > project, so I can take all the time I want.
>> > For a normal project, you usually start with an already existing
>> > library, and only modify/rewrite it if really necessary.
>> This still doesn't make sense unless the whole point of your project
>> is learning. Otherwise you'd have to write your own operating system
>> and compiler first, too. :P
>
> I disagree. For people to write file systems in other languages, a
> protocol might be more helpful than a C library.

That's twice wrong :-).

Firstly, he's not going to write it in another language. He wants to
write it in Go, and there already is a FUSE library written (natively)
in Go.

Secondly, there's pretty much nothing more universal than a C
library. Every language worth programming in has some interface to call
C libraries.


> Also, when all the operations in your file system are asynchronous,
> the synchronous interface of fuse_operations might be a problem.

I assume now you're talking about libfuse, rather than gofuse (which
would be the expected starting point). But even in libfuse, the
low-level interface is asynchronous.


Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Manlio Perillo
In reply to this post by Antonio SJ Musumeci
On Wed, Apr 13, 2016 at 6:29 PM, Antonio SJ Musumeci
<[hidden email]> wrote:
> Please!
>
> I've wanted to spend time doing the same but had gotten busy. My usecase for
> FUSE would benefit from a different userland library design (relative to
> libfuse) but needing to first half reverse engineer the protocol has kept me
> from exploring it.
>
> If I can help in any way let me know.
>

One information I may need now, for document organization, is about
the FUSE request/response model.
Are unsolicited responses supported?

Help is also appreciated (and does not require coordination) with the
mapping between UNIX system calls and FUSE requests.
This can be added as a separate page on the wiki.


Thanks  Manlio

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Manlio Perillo
In reply to this post by Nikolaus Rath
On Thu, Apr 14, 2016 at 4:42 PM, Nikolaus Rath <[hidden email]> wrote:

> On Apr 14 2016, Alejandro Exojo <alex-efHHZNW+34gk9YCHp/[hidden email]> wrote:
>> On 14 April 2016 at 09:32, Michael Zimmermann <[hidden email]> wrote:
>>>
>>> > The special thing is that there are no deadlines for a personal
>>> > project, so I can take all the time I want.
>>> > For a normal project, you usually start with an already existing
>>> > library, and only modify/rewrite it if really necessary.
>>> This still doesn't make sense unless the whole point of your project
>>> is learning. Otherwise you'd have to write your own operating system
>>> and compiler first, too. :P
>>
>> I disagree. For people to write file systems in other languages, a
>> protocol might be more helpful than a C library.
>
> That's twice wrong :-).
>
> Firstly, he's not going to write it in another language. He wants to
> write it in Go, and there already is a FUSE library written (natively)
> in Go.
>

> [...]

This is true.
However I do not simply want to write a Go implementation of the FUSE protocol.
I want to do some experiments:

1) Creation of a protocol definition that is both readable by an human
    (something like an IEFT RFC) and by a computer.
2) Generate low level implementation of the protocol *and* the documentation.
    As an example the XCB protocol definition does not contain documentation,
    so the generated API is not easy to understand unless you also happen to
    know the protocol definition.
3) Generate a test suite for the validation of the generated low level
implementation
    of the protocol.
    This can probably also be used to generate a test suite for the
validation of the
    kernel module.


Manlio

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Alejandro Exojo
In reply to this post by Nikolaus Rath
On 14 April 2016 at 16:42, Nikolaus Rath <[hidden email]> wrote:
>> I disagree. For people to write file systems in other languages, a
>> protocol might be more helpful than a C library.
>
> That's twice wrong :-).

So it's right then? Because !!wrong == right. :-)

> Firstly, he's not going to write it in another language. He wants to
> write it in Go, and there already is a FUSE library written (natively)
> in Go.

Good for them. I wasn't aware of that. But I wasn't thinking on just Go, though.

> Secondly, there's pretty much nothing more universal than a C
> library. Every language worth programming in has some interface to call
> C libraries.

That doesn't mean it's the preferred way to do it. Not having to deal
with compiling modules or C APIs in the implementation is seen often
as a good thing. I've seen "pure Perl", or "pure Ruby" or "pure JS" as
a selling point of some libraries, for example. Libraries don't end up
being idiomatic if they don't support their usual types or
conventions.

In my case, I used C++, so interfacing with C is of course dead easy,
but all the networking code is based on Qt's event loop, and I did not
see a way to integrate with fuse_loop, besides using a dedicated
thread.

So, yeah, pretty much nothing is more universal than a C API, but a
protocol is one of the things, IMHO. ;-)

>> Also, when all the operations in your file system are asynchronous,
>> the synchronous interface of fuse_operations might be a problem.
>
> I assume now you're talking about libfuse, rather than gofuse (which
> would be the expected starting point). But even in libfuse, the
> low-level interface is asynchronous.

I'll look at it again. When I did some months ago, it did not seem
very documented. Probably makes a lot of sense if you actually know
kernel programming and file systems. I have to admit that I forgot
what an inode is once I left university. :-(

Thank you!

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Werner Baumann
In reply to this post by Nikolaus Rath
Could you please stop trying to talk Manilo out of his project. What he
is intending to do would be very helpful for a lot of people, me
included.

There are good reasons not to use libfuse but to directly connect the
user space daemon to the kernel interface. Avoiding a lot of overhead
is one of them.

While the kernel interface is mostly undocumented, libfuse
documentation is not much better. Just look at all the questions on
this list.

When I added fuse support to my user space file system (davfs2) I only
got it working because I got additional information from Miklos and
studied a lot of libfuse to see how different upcalls are expected to
be served.

And I would ask all the people that are developing fuse, especially
Miklos, to support Manilo as good as they can. fuse with documentation
would really be a step forward.

Werner

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

David Sheets
In reply to this post by Manlio Perillo
Hi Manlio,

I think your project sounds interesting as I am fascinated by the FUSE
kernel interface.

You may be interested in my project, profuse <
https://github.com/dsheets/profuse >, which contains protocol bindings
to Linux FUSE 7.8. I originally started the project in early 2014 in
order to serve Linux-native file systems from MirageOS unikernels.

Looking at the /dev/fuse interface from a protocol perspective is a
trade-off and some of the pitfalls may cause issues with automated
generation of protocol transceivers:

1. Sensitivity to endianness
2. Sensitivity to errno, dirent, fcntl flag codes
3. Sensitivity to struct alignment
4. Message field disparity between Linux FUSE 7.8, OS X FUSE 7.8, and
FreeBSD FUSE 7.8 (OpenBSD appears to offer a libfuse-like interface
only)

I am currently working on a project to put profuse into production as
a trans-hypervisor protocol and it is progressing well so far. I have
another (as yet unreleased project... soon!) which provides the
necessary superprotocol and plumbing to mitigate some of the above
issues as well as a collection of C macro export libraries to deal
with the various client-specific encodings in use.

We're currently heavily focused on performance optimization and the
specific semantics of the dentry and inode caches.

Please, let me know if there is anything I can do to help you in your
work or if you think we could collaborate. I'm currently located in
Cambridge, UK if you are ever in the area.

Finally, when any FUSE developer wants to ensure that they have
implemented the POSIX file system semantics accurately, they may want
to look at SibylFS < https://sibylfs.github.io/ > which provides a
formal, non-deterministic (allows multiple correct behaviors) model of
file system semantics sufficient to check an existing file system
implementation for POSIX (or Linux-, OS X-, or FreeBSD-flavored POSIX)
compliance.

Best of luck and determination,

David Sheets

On Thu, Apr 14, 2016 at 4:29 PM, Manlio Perillo
<[hidden email]> wrote:

> On Thu, Apr 14, 2016 at 4:42 PM, Nikolaus Rath <[hidden email]> wrote:
>> On Apr 14 2016, Alejandro Exojo <alex-efHHZNW+34gk9YCHp/[hidden email]> wrote:
>>> On 14 April 2016 at 09:32, Michael Zimmermann <[hidden email]> wrote:
>>>>
>>>> > The special thing is that there are no deadlines for a personal
>>>> > project, so I can take all the time I want.
>>>> > For a normal project, you usually start with an already existing
>>>> > library, and only modify/rewrite it if really necessary.
>>>> This still doesn't make sense unless the whole point of your project
>>>> is learning. Otherwise you'd have to write your own operating system
>>>> and compiler first, too. :P
>>>
>>> I disagree. For people to write file systems in other languages, a
>>> protocol might be more helpful than a C library.
>>
>> That's twice wrong :-).
>>
>> Firstly, he's not going to write it in another language. He wants to
>> write it in Go, and there already is a FUSE library written (natively)
>> in Go.
>>
>
>> [...]
>
> This is true.
> However I do not simply want to write a Go implementation of the FUSE protocol.
> I want to do some experiments:
>
> 1) Creation of a protocol definition that is both readable by an human
>     (something like an IEFT RFC) and by a computer.
> 2) Generate low level implementation of the protocol *and* the documentation.
>     As an example the XCB protocol definition does not contain documentation,
>     so the generated API is not easy to understand unless you also happen to
>     know the protocol definition.
> 3) Generate a test suite for the validation of the generated low level
> implementation
>     of the protocol.
>     This can probably also be used to generate a test suite for the
> validation of the
>     kernel module.
>
>
> Manlio
>
> ------------------------------------------------------------------------------
> Find and fix application performance issues faster with Applications Manager
> Applications Manager provides deep performance insights into multiple tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> --
> fuse-devel mailing list
> To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Jean-Pierre André
David Sheets wrote:

[...]

> Looking at the /dev/fuse interface from a protocol perspective is a
> trade-off and some of the pitfalls may cause issues with automated
> generation of protocol transceivers:
>
> 1. Sensitivity to endianness
> 2. Sensitivity to errno, dirent, fcntl flag codes
> 3. Sensitivity to struct alignment
> 4. Message field disparity between Linux FUSE 7.8, OS X FUSE 7.8, and
> FreeBSD FUSE 7.8 (OpenBSD appears to offer a libfuse-like interface
> only)

This reminds me of an issue with OpenIndiana (and Solaris)
for 64-bit computers : for block and character devices,
major and minor are 32-bits each, so they cannot both fit
in the field rdev of struct fuse_mknod_in.

This should be tracked somewhere, so that it can be taken
into account some day.

Jean-Pierre

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Manlio Perillo
In reply to this post by David Sheets
On Fri, Apr 15, 2016 at 10:46 AM, David Sheets <[hidden email]> wrote:

> Hi Manlio,
>
> I think your project sounds interesting as I am fascinated by the FUSE
> kernel interface.
>
> You may be interested in my project, profuse <
> https://github.com/dsheets/profuse >, which contains protocol bindings
> to Linux FUSE 7.8. I originally started the project in early 2014 in
> order to serve Linux-native file systems from MirageOS unikernels.
>

Thanks.

> Looking at the /dev/fuse interface from a protocol perspective is a
> trade-off and some of the pitfalls may cause issues with automated
> generation of protocol transceivers:
>
> 1. Sensitivity to endianness
> 2. Sensitivity to errno, dirent, fcntl flag codes

These two are not a problem, since both client and server runs on the
same system.
As long as the client use standard (e.g. POSIX) errno values, the code
should be portable.

> 3. Sensitivity to struct alignment

This is a problem I have seen on a lot of code; it is not a problem
with the protocol.
Probably for laziness and to avoid copy, they read data directly into a struct.
Compare this against
https://github.com/BurntSushi/xgb/blob/master/xproto/xproto.go#L10444

> 4. Message field disparity between Linux FUSE 7.8, OS X FUSE 7.8, and
> FreeBSD FUSE 7.8 (OpenBSD appears to offer a libfuse-like interface
> only)
>

Do you have more details about this issue, or links to the source code?

> [...]

> Please, let me know if there is anything I can do to help you in your
> work or if you think we could collaborate. I'm currently located in
> Cambridge, UK if you are ever in the area.
>

I live in Italy, near Naples.

> Finally, when any FUSE developer wants to ensure that they have
> implemented the POSIX file system semantics accurately, they may want
> to look at SibylFS < https://sibylfs.github.io/ > which provides a
> formal, non-deterministic (allows multiple correct behaviors) model of
> file system semantics sufficient to check an existing file system
> implementation for POSIX (or Linux-, OS X-, or FreeBSD-flavored POSIX)
> compliance.
>

Interesting, but it seems to be useful only for filesystem implementations.
I plan to use FUSE for the implementation of a "fuzzy" filesystem: a
filesystem that can be used to test if programs use the filesystem
correctly
(see http://danluu.com/file-consistency/).

> Best of luck and determination,
>
> David Sheets
>

Thanks  Manlio

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

David Sheets
On Fri, Apr 15, 2016 at 5:20 PM, Manlio Perillo
<[hidden email]> wrote:

> On Fri, Apr 15, 2016 at 10:46 AM, David Sheets <[hidden email]> wrote:
>> Hi Manlio,
>>
>> I think your project sounds interesting as I am fascinated by the FUSE
>> kernel interface.
>>
>> You may be interested in my project, profuse <
>> https://github.com/dsheets/profuse >, which contains protocol bindings
>> to Linux FUSE 7.8. I originally started the project in early 2014 in
>> order to serve Linux-native file systems from MirageOS unikernels.
>>
>
> Thanks.
>
>> Looking at the /dev/fuse interface from a protocol perspective is a
>> trade-off and some of the pitfalls may cause issues with automated
>> generation of protocol transceivers:
>>
>> 1. Sensitivity to endianness
>> 2. Sensitivity to errno, dirent, fcntl flag codes
>
> These two are not a problem, since both client and server runs on the
> same system.
> As long as the client use standard (e.g. POSIX) errno values, the code
> should be portable.

I suppose it depends on what you mean by "same system" and if you'd
like, e.g., protocol traces to be readable at a different time on a
different system.

Unfortunately, POSIX specifies errno macros as far as I know and so
the actual values in use may not coincide unless the errno is old and
the kernel implementation used the historic value when it began. If
you are writing a FUSE protocol specification, the actual bytes to use
should somehow be specified and so you have a choice to either:

1. Specify only Linux FUSE
2. Specify the so-called FUSE protocol with an errno map between
symbols and bytes-on-the-wire (maybe implicit)

For example, on Linux, ELOOP is 40 but on OS X ELOOP is 62. Linux
ENOSYS is 38 but OS X ENOSYS is 78. Linux ESTALE is 116 but OS X
ESTALE is 70 and so on.

>> 3. Sensitivity to struct alignment
>
> This is a problem I have seen on a lot of code; it is not a problem
> with the protocol.
> Probably for laziness and to avoid copy, they read data directly into a struct.
> Compare this against
> https://github.com/BurntSushi/xgb/blob/master/xproto/xproto.go#L10444

Perhaps we have a different concept of what the protocol is and what a
specification would require. As far as I know, the protocol is
whatever the header file in use describes currently.

>> 4. Message field disparity between Linux FUSE 7.8, OS X FUSE 7.8, and
>> FreeBSD FUSE 7.8 (OpenBSD appears to offer a libfuse-like interface
>> only)
>>
>
> Do you have more details about this issue, or links to the source code?

I don't have a comprehensive location for details, unfortunately,
which is why I am interested in your project if you plan to document
the differences between the dialects. The closest I can offer is this
< https://github.com/dsheets/profuse/tree/29fb83e98d5ddc48c37264b35df43baa96b14f99/lib
> historical revision of profuse where I had attempted to capture the
differences between Linux and OS X while sharing their common features
and maintaining a good type discipline. By manually inspecting the
differences between the various _linux_7_8 and _osx_7_8 files, you can
see some of the variation. In particular, getxattr_in, setxattr_in,
and setattr_in have more fields on OS X and the attr struct has crtime
fields on OS X which changes the message structures for entry_out,
attr_out, and create_out. I haven't studied the FreeBSD implementation
in detail and OpenBSD may have moved to a protocol-based
implementation by now.

Please note, I only used that old code extensively on Linux and so it
is possible some OSXFUSE details are wrong. I would suggest reading
the OSXFUSE sources to definitively understand the differences. ;-)

>> Finally, when any FUSE developer wants to ensure that they have
>> implemented the POSIX file system semantics accurately, they may want
>> to look at SibylFS < https://sibylfs.github.io/ > which provides a
>> formal, non-deterministic (allows multiple correct behaviors) model of
>> file system semantics sufficient to check an existing file system
>> implementation for POSIX (or Linux-, OS X-, or FreeBSD-flavored POSIX)
>> compliance.
>>
>
> Interesting, but it seems to be useful only for filesystem implementations.
> I plan to use FUSE for the implementation of a "fuzzy" filesystem: a
> filesystem that can be used to test if programs use the filesystem
> correctly
> (see http://danluu.com/file-consistency/).

That sounds very interesting. We considered using SibylFS to check
program access traces however the full FS API in practice is far too
large and complex to specify formally in a small number of years. In
particular, features like mmap introduce the need to include a memory
model and potentially aliasing semantics to compute the (now huge)
potential state-space of the system. One could re-tool SibylFS to
operate on a subset of traces but many interesting real world programs
will likely be out-of-reach.

For your program validation use case, it seems like you don't actually
need a specification of the complete FUSE protocol. I think you will
want to turn off all caching and disable many (all?) of the advanced
features. The FUSE level may also obscure some details that you may
care about (compared with the syscall level) but I haven't thought
about this very much and I don't know what exactly you have in mind.
I'd be very interested to hear more about your thoughts on validating
programs that use the file system. Also, I seem to recall reading some
papers about similar topics but I'll have to dig them up...

David

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Manlio Perillo
On Fri, Apr 15, 2016 at 7:29 PM, David Sheets <[hidden email]> wrote:

> On Fri, Apr 15, 2016 at 5:20 PM, Manlio Perillo
> <[hidden email]> wrote:
> [...]
>>> Looking at the /dev/fuse interface from a protocol perspective is a
>>> trade-off and some of the pitfalls may cause issues with automated
>>> generation of protocol transceivers:
>>>
>>> 1. Sensitivity to endianness
>>> 2. Sensitivity to errno, dirent, fcntl flag codes
>>
>> These two are not a problem, since both client and server runs on the
>> same system.
>> As long as the client use standard (e.g. POSIX) errno values, the code
>> should be portable.
>
> I suppose it depends on what you mean by "same system"

The user program that access the filesystem, the OS kernel and the
FUSE server are all on the same host.

> and if you'd
> like, e.g., protocol traces to be readable at a different time on a
> different system.
>

This is a bit more complex, but not impossible IMHO.

> Unfortunately, POSIX specifies errno macros as far as I know and so
> the actual values in use may not coincide unless the errno is old and
> the kernel implementation used the historic value when it began. If
> you are writing a FUSE protocol specification, the actual bytes to use
> should somehow be specified and so you have a choice to either:
>
> 1. Specify only Linux FUSE
> 2. Specify the so-called FUSE protocol with an errno map between
> symbols and bytes-on-the-wire (maybe implicit)
>
> For example, on Linux, ELOOP is 40 but on OS X ELOOP is 62. Linux
> ENOSYS is 38 but OS X ENOSYS is 78. Linux ESTALE is 116 but OS X
> ESTALE is 70 and so on.
>

When you are on Linux, as an example, if the user program use ELOOP to
check for an error, and the FUSE server use ELOOP to report an error,
then both see the same value.
When the program is ported on FreeBSD, and recompiled, the value will
be different, but the user program and the FUSE serve will still see
the same value.

You have a portability problem only when the client and server are on
different systems.

>>> 3. Sensitivity to struct alignment
>>
>> This is a problem I have seen on a lot of code; it is not a problem
>> with the protocol.
>> Probably for laziness and to avoid copy, they read data directly into a struct.
>> Compare this against
>> https://github.com/BurntSushi/xgb/blob/master/xproto/xproto.go#L10444
>
> Perhaps we have a different concept of what the protocol is and what a
> specification would require. As far as I know, the protocol is
> whatever the header file in use describes currently.
>

The problem is that a binary protocol is not described by a C struct.
A C struct has implementation defined alignment requirements.

A binary protocol is defined by the exact layout of data fields for
each message.
Again, se the XWindow protocol implementation in xcb or xgb.

>>> 4. Message field disparity between Linux FUSE 7.8, OS X FUSE 7.8, and
>>> FreeBSD FUSE 7.8 (OpenBSD appears to offer a libfuse-like interface
>>> only)
>>>
>>
>> Do you have more details about this issue, or links to the source code?
>
> I don't have a comprehensive location for details, unfortunately,
> which is why I am interested in your project if you plan to document
> the differences between the dialects. The closest I can offer is this
> < https://github.com/dsheets/profuse/tree/29fb83e98d5ddc48c37264b35df43baa96b14f99/lib
>> historical revision of profuse where I had attempted to capture the
> differences between Linux and OS X while sharing their common features
> and maintaining a good type discipline.
> [...]

Ok, thanks.

> [...]
>> Interesting, but it seems to be useful only for filesystem implementations.
>> I plan to use FUSE for the implementation of a "fuzzy" filesystem: a
>> filesystem that can be used to test if programs use the filesystem
>> correctly
>> (see http://danluu.com/file-consistency/).
>
> [...]
>
> For your program validation use case, it seems like you don't actually
> need a specification of the complete FUSE protocol. I think you will
> want to turn off all caching and disable many (all?) of the advanced
> features.
> care about (compared with the syscall level) but I haven't thought
> about this very much and I don't know what exactly you have in mind.

It's the same for me.
I first need to learn how FUSE works and do some tests to learn how
POSIX filesystem calls are mapped to FUSE.

> [...]

Manlio

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

David Sheets
On Fri, Apr 15, 2016 at 7:47 PM, Manlio Perillo
<[hidden email]> wrote:
> On Fri, Apr 15, 2016 at 7:29 PM, David Sheets <[hidden email]> wrote:
>> On Fri, Apr 15, 2016 at 5:20 PM, Manlio Perillo
> [...]
>> and if you'd
>> like, e.g., protocol traces to be readable at a different time on a
>> different system.
>>
>
> This is a bit more complex, but not impossible IMHO.

I agree that it is easily possible but it creates the need to specify
the FUSE dialect and read or negotiate that dialect in interpreters.
I'm very interested in collaborating on standard protocols and formats
to deal with this issue. I'd like to see the equivalent of pcap for
FUSE.

>> Unfortunately, POSIX specifies errno macros as far as I know and so
>> the actual values in use may not coincide unless the errno is old and
>> the kernel implementation used the historic value when it began. If
>> you are writing a FUSE protocol specification, the actual bytes to use
>> should somehow be specified and so you have a choice to either:
>>
>> 1. Specify only Linux FUSE
>> 2. Specify the so-called FUSE protocol with an errno map between
>> symbols and bytes-on-the-wire (maybe implicit)
>>
>> For example, on Linux, ELOOP is 40 but on OS X ELOOP is 62. Linux
>> ENOSYS is 38 but OS X ENOSYS is 78. Linux ESTALE is 116 but OS X
>> ESTALE is 70 and so on.
>>
>
> When you are on Linux, as an example, if the user program use ELOOP to
> check for an error, and the FUSE server use ELOOP to report an error,
> then both see the same value.
> When the program is ported on FreeBSD, and recompiled, the value will
> be different, but the user program and the FUSE serve will still see
> the same value.
>
> You have a portability problem only when the client and server are on
> different systems.

Yes, I see this as an issue when specifying the protocol as I believe
the protocol specification should be able to describe the bit patterns
exchanged. In the common case, everything lines up as you say. I think
the FUSE protocol (warts and all) is the closest we have to a data
type for the Linux VFS/FS interface (9p2000.L is the only other
similar thing I know of and it is not nearly as well-developed or
complete). I'd like to see multiple VMs on the same hypervisor on the
same hardware share a file system directly with minimal in-kernel
conversion. FUSE was not meant to be a portable protocol but it
*could* be specified to be one with slightly more metadata.

>>>> 3. Sensitivity to struct alignment
>>>
>>> This is a problem I have seen on a lot of code; it is not a problem
>>> with the protocol.
>>> Probably for laziness and to avoid copy, they read data directly into a struct.
>>> Compare this against
>>> https://github.com/BurntSushi/xgb/blob/master/xproto/xproto.go#L10444
>>
>> Perhaps we have a different concept of what the protocol is and what a
>> specification would require. As far as I know, the protocol is
>> whatever the header file in use describes currently.
>>
>
> The problem is that a binary protocol is not described by a C struct.
> A C struct has implementation defined alignment requirements.
>
> A binary protocol is defined by the exact layout of data fields for
> each message.
> Again, se the XWindow protocol implementation in xcb or xgb.

I agree with you that exact layout is /necessary/ to specify the
binary protocol. I am suggesting that the current protocol that a
given system uses is implicitly defined by the local equivalent of <
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=refs/tags/v4.6-rc3
>. This file should have structs with sizes in multiples of 8 and
names in dirent returns should be padded to multiples of 8. I believe
all of the FUSE implementations obey this and so you are right that
struct alignment should not be an issue. I'm sorry I misremembered the
issue. :-)

> [...]

David

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

Antonio SJ Musumeci

"9p2000.L is the only other similar thing I know of and it is not nearly as well-developed or complete)"

I'm not sure that's true. It's the basis for VirtIO is it not? And isn't FUSE more or less a reimplementation of the 9p idea? If you compare the two protocols they are rather similar.

On Apr 16, 2016 1:10 PM, "David Sheets" <[hidden email]> wrote:
On Fri, Apr 15, 2016 at 7:47 PM, Manlio Perillo
<[hidden email]> wrote:
> On Fri, Apr 15, 2016 at 7:29 PM, David Sheets <[hidden email]> wrote:
>> On Fri, Apr 15, 2016 at 5:20 PM, Manlio Perillo
> [...]
>> and if you'd
>> like, e.g., protocol traces to be readable at a different time on a
>> different system.
>>
>
> This is a bit more complex, but not impossible IMHO.

I agree that it is easily possible but it creates the need to specify
the FUSE dialect and read or negotiate that dialect in interpreters.
I'm very interested in collaborating on standard protocols and formats
to deal with this issue. I'd like to see the equivalent of pcap for
FUSE.

>> Unfortunately, POSIX specifies errno macros as far as I know and so
>> the actual values in use may not coincide unless the errno is old and
>> the kernel implementation used the historic value when it began. If
>> you are writing a FUSE protocol specification, the actual bytes to use
>> should somehow be specified and so you have a choice to either:
>>
>> 1. Specify only Linux FUSE
>> 2. Specify the so-called FUSE protocol with an errno map between
>> symbols and bytes-on-the-wire (maybe implicit)
>>
>> For example, on Linux, ELOOP is 40 but on OS X ELOOP is 62. Linux
>> ENOSYS is 38 but OS X ENOSYS is 78. Linux ESTALE is 116 but OS X
>> ESTALE is 70 and so on.
>>
>
> When you are on Linux, as an example, if the user program use ELOOP to
> check for an error, and the FUSE server use ELOOP to report an error,
> then both see the same value.
> When the program is ported on FreeBSD, and recompiled, the value will
> be different, but the user program and the FUSE serve will still see
> the same value.
>
> You have a portability problem only when the client and server are on
> different systems.

Yes, I see this as an issue when specifying the protocol as I believe
the protocol specification should be able to describe the bit patterns
exchanged. In the common case, everything lines up as you say. I think
the FUSE protocol (warts and all) is the closest we have to a data
type for the Linux VFS/FS interface (9p2000.L is the only other
similar thing I know of and it is not nearly as well-developed or
complete). I'd like to see multiple VMs on the same hypervisor on the
same hardware share a file system directly with minimal in-kernel
conversion. FUSE was not meant to be a portable protocol but it
*could* be specified to be one with slightly more metadata.

>>>> 3. Sensitivity to struct alignment
>>>
>>> This is a problem I have seen on a lot of code; it is not a problem
>>> with the protocol.
>>> Probably for laziness and to avoid copy, they read data directly into a struct.
>>> Compare this against
>>> https://github.com/BurntSushi/xgb/blob/master/xproto/xproto.go#L10444
>>
>> Perhaps we have a different concept of what the protocol is and what a
>> specification would require. As far as I know, the protocol is
>> whatever the header file in use describes currently.
>>
>
> The problem is that a binary protocol is not described by a C struct.
> A C struct has implementation defined alignment requirements.
>
> A binary protocol is defined by the exact layout of data fields for
> each message.
> Again, se the XWindow protocol implementation in xcb or xgb.

I agree with you that exact layout is /necessary/ to specify the
binary protocol. I am suggesting that the current protocol that a
given system uses is implicitly defined by the local equivalent of <
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?id=refs/tags/v4.6-rc3
>. This file should have structs with sizes in multiples of 8 and
names in dirent returns should be padded to multiples of 8. I believe
all of the FUSE implementations obey this and so you are right that
struct alignment should not be an issue. I'm sorry I misremembered the
issue. :-)

> [...]

David

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: FUSE protocol documentation

David Sheets
On Sat, Apr 16, 2016 at 1:23 PM, Antonio SJ Musumeci
<[hidden email]> wrote:
> "9p2000.L is the only other similar thing I know of and it is not nearly as
> well-developed or complete)"
>
> I'm not sure that's true. It's the basis for VirtIO is it not?

Yes, 9p2000.L is the basis for the virtio file system pass-through
solution, VirtFS. This was an interesting technical choice.

I've not worked extensively with 9p2000.L but I have studied it in
order to understand how it compares with FUSE.

Here is my understanding:

9p2000.L in Linux supports many types of transport (unix socket, tcp,
fd, virtio, and rdma) as 9p2000.* was designed to be a network
protocol.

9p2000.L is more flexible in stat and statfs return structures by
allowing the implementation to dynamically determine the device ID,
file system type, ID, and name.

I believe 9p2000.L also supports fstat on unlinked files and FUSE 7.24
as implemented in Linux 4.5 does not but I have not attempted to do
this with 9p2000.L or recently with FUSE (I've mostly used 7.8 so
far)... does anyone know the status of FUSE_FILE_OPS/FUSE_GETTATTR_FH?

FUSE supports fallocate used by, e.g., MySQL.

FUSE supports renameat2.

FUSE supports dentry and inode cache TTLs on a per-object basis.

FUSE supports server-pushed notifications for dentry and inode cache control.

FUSE provides cache-control mechanisms and seekability on a per-file
basis via open response flags.

FUSE supports changing open flags after open using fcntl.

FUSE supports ioctl(2) and poll(2).

FUSE supports bmap.

FUSE supports mandatory locking and BSD flock if that's your thing <
http://0pointer.de/blog/projects/locking.html >.

FUSE allows file systems to opt into resolution of "." and ".." with
the FUSE_EXPORT_SUPPORT flag.

FUSE is more flexible in handling of umask application (can defer it
to the server for ACL use).

FUSE provides the kernel thread ID for each request.

Finally, the FUSE development model is much more conducive to updates
as the Linux VFS evolves precisely because the FUSE protocol is not
intended to be portable or standardized! Also, Miklos and the other
FUSE developers are awesome. By comparsion 9p2000.L does not have
anything like the history of FUSE improvements although VirtFS might
change this and we may see 9p2000.L2 and 9p2000.L3, etc.

> And isn't FUSE more or less a reimplementation of the 9p idea? If you compare the two
> protocols they are rather similar.

I would say 9p2000.L is similar to FUSE in that it tries to match the
Linux VFS closely. 9p, 9p2000, and 9p2000.u were not designed to even
be POSIX compliant let alone support obscure VFS features.

In the more general sense of running file servers in user space or
mounting applications as synthetic file systems, Plan 9's 9p did
extensively use and popularize this idea. I believe that OSes like
ITS, protocols like NFS, and later microkernels like Mach were
actually the first systems to feature these concepts. I see FUSE as a
specialized, partial microkernel-esque feature of Linux allowing
independent file system services to offer a nearly Linux-native
virtual file system.

Has anyone booted from a FUSE mount? I think it should be possible but
I haven't tried, yet...

Hope that helps,

David

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel