Passing struct fuse_file_info to truncate?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Passing struct fuse_file_info to truncate?

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I'm trying to handle a ftruncate() call with my fuse truncate()
function. ftruncate() can only be called with on an open file, but
because of the prototype "int (*truncate) (const char *, off_t);" i
cannot access "struct fuse_file_info" to check, if the file is already
opened.

Maybe it's usefull to pass the struct fuse_file_info to truncate?

Greetings, Jens

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDAd+uBoFc9p8JILsRAi2MAKDbCoCk3rg3u7ndr56Zd8ayoi0j6gCdFfgJ
KouOJ5abid9Q8RwOSeYGsAU=
=V1eh
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

Miklos Szeredi
> I'm trying to handle a ftruncate() call with my fuse truncate()
> function. ftruncate() can only be called with on an open file, but
> because of the prototype "int (*truncate) (const char *, off_t);" i
> cannot access "struct fuse_file_info" to check, if the file is already
> opened.
>
> Maybe it's usefull to pass the struct fuse_file_info to truncate?

This is not possible unfortunately: the open file on which the
ftruncate() operation was performed is simply not available to the
filesystem implementation in the Linux kernel.  Same is true for
fstat(), fchown(), fchmod().

Miklos


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Miklos Szeredi wrote at 16.08.2005 15:22:

>> Maybe it's usefull to pass the struct fuse_file_info to truncate?
>
> This is not possible unfortunately: the open file on which the
> ftruncate() operation was performed is simply not available to the
> filesystem implementation in the Linux kernel.

OK. Good reason. ;)

So how can I determine, wether a file is currently open within the fuse
truncate() function? I've tried to store this information globaly when
open() is called, but when a file is opened more than once I can't
figure out in truncate() which one I should truncate...

Thanks for your help and greetings, Jens

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDAe8lBoFc9p8JILsRAitFAKDb5LbKKD15VTbjiXMt5bXiyAVu/wCbBbXU
0IU2KSaHRbyo/sSCjzdWuWE=
=9rC3
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

Miklos Szeredi
> OK. Good reason. ;)
>
> So how can I determine, wether a file is currently open within the fuse
> truncate() function? I've tried to store this information globaly when
> open() is called, but when a file is opened more than once I can't
> figure out in truncate() which one I should truncate...

Does it matter?  If they all refer to the same underlying file, then
it shouldn't.

This is just an optimization, right?  The simple solution is to do
open/ftruncate/close in the truncate method (which you'll need anyway,
if the file isn't open).

Miklos


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Miklos Szeredi wrote at 16.08.2005 16:02:

>>So how can I determine, wether a file is currently open within the fuse
>>truncate() function? I've tried to store this information globaly when
>>open() is called, but when a file is opened more than once I can't
>>figure out in truncate() which one I should truncate...
>
> Does it matter?  If they all refer to the same underlying file, then
> it shouldn't.

It matters to me, because I was a little bit lazy. ;)

My FS gets a file from a server to a filehandle in open() already.
Read and write calls are simple handled by pread()/pwrite(). The file
is put back to the server in release(), when the data has changed.

O_TRUNC and truncate() are working fine, but ftruncate() doesn't work
in situations like this:

1. open()      - get the file to the filehandle
2. write()     - does only effect the filehandle
3. ftruncate() - get the file again, truncates it, put is back
4. close()     - release() writes the filehandle back and overwrites 3.

So my 2nd idea was to store informations about all currently opened
files globaly and let truncate() check, wether a file is already open
and than use ftruncate() with the filehandle. But then I'd need to
which file I should truncate if the file is opened more than once.

Greetings, Jens

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDAiHTBoFc9p8JILsRAlY/AKDjzAl8JmoC0T9XdY/lid9n7hT3oACfWAaq
81y4wbFz3/ksoROR7qnRybA=
=rcbd
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

Miklos Szeredi
> It matters to me, because I was a little bit lazy. ;)
>
> My FS gets a file from a server to a filehandle in open() already.
> Read and write calls are simple handled by pread()/pwrite(). The file
> is put back to the server in release(), when the data has changed.
>
> O_TRUNC and truncate() are working fine, but ftruncate() doesn't work
> in situations like this:
>
> 1. open()      - get the file to the filehandle
> 2. write()     - does only effect the filehandle
> 3. ftruncate() - get the file again, truncates it, put is back
> 4. close()     - release() writes the filehandle back and overwrites 3.

OK, I understand the problem now.

> So my 2nd idea was to store informations about all currently opened
> files globaly and let truncate() check, wether a file is already open
> and than use ftruncate() with the filehandle. But then I'd need to
> which file I should truncate if the file is opened more than once.

But if file is opened more than once this scheme is broken anyway:

process 1           process 2

open()
                    open()
write()
                    write()
release()
                    release()


Now the write by process 1 is lost.

So if you buffer data, you should have a global buffer, and write back
on the last release on the file (or at least the last release for a
file open for writing).

Miklos


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Miklos,

>>O_TRUNC and truncate() are working fine, but ftruncate() doesn't work
>>in situations like this:
>>
>>1. open()      - get the file to the filehandle
>>2. write()     - does only effect the filehandle
>>3. ftruncate() - get the file again, truncates it, put is back
>>4. close()     - release() writes the filehandle back and overwrites 3.
>
> OK, I understand the problem now.

My current workaround is to put the file back to the server in the
write() method. ftruncate() reads the latest data and can do it's work.
The other way around, if the sequence was open(), ftruncate(), write()
I must ensure , that write() rereads the data, because it was changed
by truncate() after open()... Note nice, but it seems to work.

I had another idea: Would it be (technical and/or theoretical) possible
to have a fuse truncate() function for truncate() and O_TRUNC as so far
and an additional ftruncate() with "struct fuse_file_info"? Because
when ftruncate() is called the file must be opened before and it should
be possible to pass fuse_file_info to ftruncate(). Just my 2 cent. :-)

> But if file is opened more than once this scheme is broken anyway:
>
> process 1           process 2
>
> open()
>                     open()
> write()
>                     write()
> release()
>                     release()
>
>
> Now the write by process 1 is lost.

That's a problem of the accessing programs not the FS. I could do this
(without locking) with every common FS like ext3.

Greetings, Jens

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDAvBUBoFc9p8JILsRAol7AJ9WfJeUfHuyAF8pfh7896UBSrNbRACgkmzq
v6cfa9GewQyU+UDJTYxhYsg=
=2uvw
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

Miklos Szeredi
> My current workaround is to put the file back to the server in the
> write() method. ftruncate() reads the latest data and can do it's work.
> The other way around, if the sequence was open(), ftruncate(), write()
> I must ensure , that write() rereads the data, because it was changed
> by truncate() after open()... Note nice, but it seems to work.
>
> I had another idea: Would it be (technical and/or theoretical) possible
> to have a fuse truncate() function for truncate() and O_TRUNC as so far
> and an additional ftruncate() with "struct fuse_file_info"? Because
> when ftruncate() is called the file must be opened before and it should
> be possible to pass fuse_file_info to ftruncate(). Just my 2 cent. :-)

Theoretically possible, technically (without modifying the kernel
internals) it is not.  I'll think about adding support for this into
the kernel.

> > But if file is opened more than once this scheme is broken anyway:
> >
> > process 1           process 2
> >
> > open()
> >                     open()
> > write()
> >                     write()
> > release()
> >                     release()
> >
> >
> > Now the write by process 1 is lost.
>
> That's a problem of the accessing programs not the FS. I could do this
> (without locking) with every common FS like ext3.

No you couldn't.  The kernel does buffering on an inode basis not on
an open-file basis.  Not to be confused by buffering in stdio, which
is a per FILE thing.

Miklos


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

jens m. noedler
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Miklos Szeredi wrote at 08/17/2005 12:26 PM:

>> I had another idea: Would it be (technical and/or theoretical)
>> possible to have a fuse truncate() function for truncate() and
>> O_TRUNC as so far and an additional ftruncate() with "struct
>> fuse_file_info"? Because when ftruncate() is called the file must
>> be opened before and it should be possible to pass fuse_file_info
>> to ftruncate(). Just my 2 cent. :-)
>
> Theoretically possible, technically (without modifying the kernel
> internals) it is not.

I don't know if this would be a requested and meaningful feature
beside Fuse, but a nice one to have.

> I'll think about adding support for this into the kernel.

You _will_ think or you do already? ;)

If it would work, implementing Fuse filesystems would be easy if you
have to access remote files and don't have a socks API, because you
could just get the file to a filehandle and deal with it, as I tried.

Thanks and bye, Jens

- --
jens m. noedler
  [hidden email]
  pgp: 0x9f0920bb
  http://noedler.de

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDA10MBoFc9p8JILsRAmBqAKCy4A2HneTO4YL/neNP4nnyY/VxWgCfUthq
pExpjBC1xOamiUxflTuxiR0=
=6uSt
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Passing struct fuse_file_info to truncate?

Miklos Szeredi
> >> I had another idea: Would it be (technical and/or theoretical)
> >> possible to have a fuse truncate() function for truncate() and
> >> O_TRUNC as so far and an additional ftruncate() with "struct
> >> fuse_file_info"? Because when ftruncate() is called the file must
> >> be opened before and it should be possible to pass fuse_file_info
> >> to ftruncate(). Just my 2 cent. :-)
> >
> > Theoretically possible, technically (without modifying the kernel
> > internals) it is not.
>
> I don't know if this would be a requested and meaningful feature
> beside Fuse, but a nice one to have.

Probably not, otherwise it would already exist.

> > I'll think about adding support for this into the kernel.
>
> You _will_ think or you do already? ;)

I'm thinking hard ;)

The solution is actually not particularly complex, the 'struct file'
pointer just has to be passed down a long call sequence.  However this
would be breaking internal APIs, which kernel developers are not
always enthusiastic about (understadibly).  So the hard problem is
actually to sell the idea.

And since I haven't even been able to sell the FUSE thing, I don't yet
have much chance success.  If FUSE is merged in mainline, then I can
start submitting more intrusive patches such as this.

> If it would work, implementing Fuse filesystems would be easy if you
> have to access remote files and don't have a socks API, because you
> could just get the file to a filehandle and deal with it, as I tried.

There's a more convincing argument for an ftruncate() method: without
it a remote filesystem (such as sshfs), which is served by an
unprivileged userspace process, can't implement "perfect" semantics
for the following (rarly used) family of cases:

   fd = open("/mnt/fuse/testfile", O_WRONLY);
   fchmod(fd, 0444);
   ftruncate(fd, 1000);
   close(fd);

Note, how the file is changed to have only read permission
(-r--r--r--), but is opened for writing.  So ftruncate() succeeds,
while truncate() fails.

There's no way to solve this without races if there's no ftruncate()
filesystem method.

Miklos


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel