open() and O_TRUNC and structs in fi->fh

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

open() and O_TRUNC and structs in fi->fh

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

According to fuse.h comments, the O_TRUNC flag is not passed to
fi->flags. But how can I determine, if a file was opened like only with
"O_WRONLY" or with "O_WRONLY | O_TRUNC"? I tested sshfs and it works as
 if it knows, wether it should truncate the file or not. How did you do
that?


Another question: Often fi->fh is used to store filesystem specific
structs with a (ugly) cast like "fi->fh = (unsigned long)my_struct;".
Wouldn't it be nicer to have an additional "void *userdata"-field in
struct fuse_file_info?


Thanks and greetings, Jens

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFC/GQRBoFc9p8JILsRAt33AJsEwjll54Efwt0hxvsUUSRS9nLAGACgwih+
K5qxZmrRtkfZE+H8jNX2SQo=
=LnHo
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: open() and O_TRUNC and structs in fi->fh

Miklos Szeredi
> According to fuse.h comments, the O_TRUNC flag is not passed to
> fi->flags. But how can I determine, if a file was opened like only with
> "O_WRONLY" or with "O_WRONLY | O_TRUNC"? I tested sshfs and it works as
>  if it knows, wether it should truncate the file or not. How did you do
> that?

The kernel separates the creation, the permission checking, the
truncation and the actual opening of the file into separate filesystem
methods.  So an open(O_WRONLY | O_TRUNC) will first call truncate()
and then open().

For truncate this is OK.  With creation, there's a slight problem with
this.

For example the call open("xy", O_WRONLY | O_CREAT | O_EXCL, 0400)
will always fail with fusexmp or sshfs, since the created file doesn't
have write permission.

This is being worked on (it needs VFS side support as well), so next
release may have a create() method that can be used to do an atomic
create/open operation.

> Another question: Often fi->fh is used to store filesystem specific
> structs with a (ugly) cast like "fi->fh = (unsigned long)my_struct;".
> Wouldn't it be nicer to have an additional "void *userdata"-field in
> struct fuse_file_info?

Yes, well, the 'fh' field could be a union.  But IMO the better (and
more type-safe) soltion is to encapsulate the casting into two inline
functions:

static inline struct my_struct *get_my_struct(struct fuse_file_info *fi)
{
        return (struct my_struct *) fi->fh;
}

static inline void set_my_struct(struct fuse_file_info *fi, struct my_struct *s)
{
        fi->fh = (unsigned long) s;
}

Miklos


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: open() and O_TRUNC and structs in fi->fh

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Miklos Szeredi wrote at 08/12/2005 12:28 PM:

>>According to fuse.h comments, the O_TRUNC flag is not passed to
>>fi->flags. But how can I determine, if a file was opened like only with
>>"O_WRONLY" or with "O_WRONLY | O_TRUNC"? I tested sshfs and it works as
>> if it knows, wether it should truncate the file or not. How did you do
>>that?
>
> The kernel separates the creation, the permission checking, the
> truncation and the actual opening of the file into separate filesystem
> methods.  So an open(O_WRONLY | O_TRUNC) will first call truncate()
> and then open().

Thanks for opening my eyes.  My filesystems now behaves a little bit
more straight forward. ;)

I missed this point because there is no comment about this and no
standard debug output like "TRUNCATE[...]".  Maybe you can add this
debug output in fuse.c and a comment in fuse.h like: "If O_TRUNC is
passed to open() or the accessing program calls ftruncate() or
truncate() the fuse function truncate() is called."

>>Another question: Often fi->fh is used to store filesystem specific
>>structs with a (ugly) cast like "fi->fh = (unsigned long)my_struct;".
>>Wouldn't it be nicer to have an additional "void *userdata"-field in
>>struct fuse_file_info?
>
> Yes, well, the 'fh' field could be a union.

No. Use the fh _only_ for the file handle and userdata for the struct.

/** Information about open files */
struct fuse_file_info {
    /** Open flags.  Available in open() and release() */
    int flags;

    /** File handle.  May be filled in by filesystem in open().
        Available in all other file operations */
    unsigned long fh;

    /** In case of a write operation indicates if this was caused by a
        writepage */
    int writepage;

    /** For a filesystem specific data (for example a struct) */
    void *userdata;
};

That would also have the nice effect, that the debug output like
"OPEN[134698600]" would alway provide the real number of the file
handle instead the long int from the cast of a struct. :-)

> But IMO the better (and more type-safe) soltion is to encapsulate the
> casting into two inline functions: [...]
>
> static inline void set_my_struct(
>              struct fuse_file_info *fi, struct my_struct *s)
> {
> fi->fh = (unsigned long) s;
> }

Doesn't fix this (to me: ugly) cast from a struct to a long unsigned...

Greetings and thanks for your fast reply and patience with fuse
beginners! Jens

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFC/KoSBoFc9p8JILsRArfWAKCjAt23nCieShvc0kzhCN2v7GGbXACfeZaj
EK3exeinh9brwYJvUg0rmGA=
=BAAH
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: open() and O_TRUNC and structs in fi->fh

Valient Gough
On Friday 12 August 2005 15:54, jens m. noedler wrote:

> > But IMO the better (and more type-safe) soltion is to encapsulate the
> > casting into two inline functions: [...]
> >
> > static inline void set_my_struct(
> >              struct fuse_file_info *fi, struct my_struct *s)
> > {
> > fi->fh = (unsigned long) s;
> > }
>
> Doesn't fix this (to me: ugly) cast from a struct to a long unsigned...

Well, the cast is only if you choose to go that route.  If you don't like the
cast, you can write your code in a way that doesn't need it at all!  For
example, you could choose to store an integer offset into an array of
struct's that you keep locally.  But storing the pointer itself is an
optimization that avoids any extra storage and lookups.

Adding a "void *userdata" field wouldn't get rid of that cast, it would just
move it to libfuse.  There is no reason to pass the extra information to the
kernel, so libfuse would have to keep track of a struct containing the  int
and void*  and do the cast before calling the user code.  This adds
unnecessary overhead, IMHO.

So, I don't think the interface forces you to make ugly casts.  So if you
don't like them, don't use them! :-)

regards,
Valient


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: open() and O_TRUNC and structs in fi->fh

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

Valient Gough write at 08/12/2005 04:20 PM:

>> Doesn't fix this (to me: ugly) cast from a struct to a long
>> unsigned...
>
> Well, the cast is only if you choose to go that route.  If you don't
>  like the cast, you can write your code in a way that doesn't need
> it at all!

Yes, I know. :-)

> For example, you could choose to store an integer offset into an
> array of struct's that you keep locally.  But storing the pointer
> itself is an optimization that avoids any extra storage and lookups.

Indeed. But that would be the same with a *userdata field.

> Adding a "void *userdata" field wouldn't get rid of that cast, [...]

But storing a struct as a void pointer is more common that storing it
as a unsigned long, isn't it?


Another question: How do you test your fuse filesystems? I've been
looking around and tried iozone [1]. With iozone one can perform the
following file operations: Read, write, re-read, re-write, read
backwards, read strided, fread, fwrite, random read/write, pread/pwrite
variants, aio_read, aio_write, mmap. Looks nice to me.

Bye, Jens

[1] http://www.iozone.org/

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFC/MOlBoFc9p8JILsRAsouAJ9UAIvq90yPt3PbdZsz14UFVy8FywCeI9eP
Lk2lH9du9ZZ4k0jAJRWsThE=
=V/oy
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: open() and O_TRUNC and structs in fi->fh

Valient Gough
On Friday 12 August 2005 17:43, jens m. noedler wrote:

> Another question: How do you test your fuse filesystems? I've been
> looking around and tried iozone [1]. With iozone one can perform the
> following file operations: Read, write, re-read, re-write, read
> backwards, read strided, fread, fwrite, random read/write, pread/pwrite
> variants, aio_read, aio_write, mmap. Looks nice to me.

Some things I run for testing from time to time:

- bonnie++ : pretty good test if you let it run over a couple GB of data.. It
also does a threaded-seek performance test which is a good test of threaded
access (although just read, not write).
- rsync : this does lots of writes and rename operations, running multiple
rsyncs at the same time on the same filesystem is pretty demanding.
- fsx-linux : I haven't run this in a long time, but a long long time ago it
was responsible for finding some bugs in my filesystem (and in fuse).

I also have some perl scripts to test very unlikely edge cases that are too
hard to trigger reliably with other programs, but those edge cases are
specific to my filesystem (like truncation related tests since my filesystem
is block-oriented), and some of the tests aren't really necessary anymore
(like tests related to renaming open files or deleting open files, etc).

I also run larger applications regularly working on data in my filesystem
(KMail, Acroread, StarOffice), but those don't usually push it as hard as the
ones I listed above..

regards,
Valient



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel