setting inodes in the highlevel API

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

setting inodes in the highlevel API

Antonio SJ Musumeci
libfuse 2.9.x

When using "use_ino" there's seems to be some oddness.

If in readdir st_ino is set to 0, the value provided by getattr appears to be ignored.
If in readdir st_ino is set to not-zero (the value intended, or just 1), the value provided by getattr overwrites it.

Does it cause problems to have readdir and getattr provide different values? That is... if I simply set the value to 1 in readdir and did my inode computation in getattr would that be alright?

Will it cause problems (perhaps in the cache) if the value returned by getattr changed?

Thanks.

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: setting inodes in the highlevel API

Nikolaus Rath
On Dec 11 2016, Antonio SJ Musumeci <[hidden email]> wrote:
> libfuse 2.9.x
>
> When using "use_ino" there's seems to be some oddness.
>
> If in readdir st_ino is set to 0, the value provided by getattr appears to
> be ignored.

I haven't checked the code, but I think using 0 as inode is generally a
bad idea. I'd only use inodes >= FUSE_ROOT_ID (which is one).

> If in readdir st_ino is set to not-zero (the value intended, or just 1),
> the value provided by getattr overwrites it.

That, I think makes sense. Something has to take precedence, and in some
sense getattr is more specific...

> Does it cause problems to have readdir and getattr provide different
> values?

Again, I haven't checked the code but yes, I would expect that to cause
problems. If the inode changes, this means that the file has been
removed and recreated.

> That is... if I simply set the value to 1 in readdir and did my
> inode computation in getattr would that be alright?

No, that would probably be a bad idea.


Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: setting inodes in the highlevel API

Antonio SJ Musumeci
Thanks. I'll look at the kernel code. There's nothing of interest in libfuse.

On Tue, Dec 13, 2016 at 1:31 PM, Nikolaus Rath <[hidden email]> wrote:
On Dec 11 2016, Antonio SJ Musumeci <[hidden email]> wrote:
> libfuse 2.9.x
>
> When using "use_ino" there's seems to be some oddness.
>
> If in readdir st_ino is set to 0, the value provided by getattr appears to
> be ignored.

I haven't checked the code, but I think using 0 as inode is generally a
bad idea. I'd only use inodes >= FUSE_ROOT_ID (which is one).

> If in readdir st_ino is set to not-zero (the value intended, or just 1),
> the value provided by getattr overwrites it.

That, I think makes sense. Something has to take precedence, and in some
sense getattr is more specific...

> Does it cause problems to have readdir and getattr provide different
> values?

Again, I haven't checked the code but yes, I would expect that to cause
problems. If the inode changes, this means that the file has been
removed and recreated.

> That is... if I simply set the value to 1 in readdir and did my
> inode computation in getattr would that be alright?

No, that would probably be a bad idea.


Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: setting inodes in the highlevel API

Antonio SJ Musumeci
I take that back.

When use_ino isn't used fill_dir sets stbuf.st_ino = FUSE_UNKNOWN_INO == 0xFFFFFFFF.

I'm not able to find that special value used anywhere in the kernel code nor outside the highlevel libfuse code for anything interesting. Appears that so long as the value is !zero getattr overwrites it. Speaking of which where does that happen?

Perhaps more perplexing is that when the inode is 0 (with use_ino) running in debug mode shows that readdir returns successfully with the same amount of data that I see when leave libfuse to generate the inodes however nothing comes through on the caller of readdir's side (readdir returns NULL). Seems like libfuse is doing some sort of post check on the kernel data? Or something post readdir isn't handling it well?

On Tue, Dec 13, 2016 at 2:01 PM, Antonio SJ Musumeci <[hidden email]> wrote:
Thanks. I'll look at the kernel code. There's nothing of interest in libfuse.

On Tue, Dec 13, 2016 at 1:31 PM, Nikolaus Rath <[hidden email]> wrote:
On Dec 11 2016, Antonio SJ Musumeci <[hidden email]> wrote:
> libfuse 2.9.x
>
> When using "use_ino" there's seems to be some oddness.
>
> If in readdir st_ino is set to 0, the value provided by getattr appears to
> be ignored.

I haven't checked the code, but I think using 0 as inode is generally a
bad idea. I'd only use inodes >= FUSE_ROOT_ID (which is one).

> If in readdir st_ino is set to not-zero (the value intended, or just 1),
> the value provided by getattr overwrites it.

That, I think makes sense. Something has to take precedence, and in some
sense getattr is more specific...

> Does it cause problems to have readdir and getattr provide different
> values?

Again, I haven't checked the code but yes, I would expect that to cause
problems. If the inode changes, this means that the file has been
removed and recreated.

> That is... if I simply set the value to 1 in readdir and did my
> inode computation in getattr would that be alright?

No, that would probably be a bad idea.


Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: setting inodes in the highlevel API

Nikolaus Rath
On Dec 13 2016, Antonio SJ Musumeci <[hidden email]> wrote:
> I take that back.
>
> When use_ino isn't used fill_dir sets stbuf.st_ino = FUSE_UNKNOWN_INO ==
> 0xFFFFFFFF.
>
> I'm not able to find that special value used anywhere in the kernel code
> nor outside the highlevel libfuse code for anything interesting. Appears
> that so long as the value is !zero getattr overwrites it. Speaking of which
> where does that happen?

I think overwriting the inode used by the high-level API must happen
only if getattr() is called as a result of a low-level lookup() request,
but not for a low-level getattr() request (because in that case the
kernel already knows the inode).

So I'd expect the code to be somewhere in fuse_lib_lookup(). The only
candidate in there is the call to lookup_path(), but this in turn calls
fuse_lib_getattr() - which must not overwrite the inode.

In other words, I am not sure how exactly use_ino is supposed to work,
nor how it is actually working at the moment.

> Perhaps more perplexing is that when the inode is 0 (with use_ino) running
> in debug mode shows that readdir returns successfully with the same amount
> of data that I see when leave libfuse to generate the inodes

Well, that is expected. The data returned to the kernel has always the
same length, no matter if the inode is defined by your filesystem or by
the high-level API internally.

Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: setting inodes in the highlevel API

Antonio SJ Musumeci
The little documentation I've found[0] says that the inode doesn't need to be unique which implies that the value may not really matter to the system.

I remember a while back noticing that setting dirent.d_type only mattered if the followup stat failed. I would suspect that is also true for the d_ino. Let me test that.

[0] fuse.h reads
/**
* Honor the st_ino field in the functions getattr() and
* fill_dir(). This value is used to fill in the st_ino field
* in the stat(2), lstat(2), fstat(2) functions and the d_ino
* field in the readdir(2) function. The filesystem does not
* have to guarantee uniqueness, however some applications
* rely on this value being unique for the whole filesystem.
*/

On Thu, Dec 15, 2016 at 11:47 AM, Nikolaus Rath <[hidden email]> wrote:
On Dec 13 2016, Antonio SJ Musumeci <[hidden email]> wrote:
> I take that back.
>
> When use_ino isn't used fill_dir sets stbuf.st_ino = FUSE_UNKNOWN_INO ==
> 0xFFFFFFFF.
>
> I'm not able to find that special value used anywhere in the kernel code
> nor outside the highlevel libfuse code for anything interesting. Appears
> that so long as the value is !zero getattr overwrites it. Speaking of which
> where does that happen?

I think overwriting the inode used by the high-level API must happen
only if getattr() is called as a result of a low-level lookup() request,
but not for a low-level getattr() request (because in that case the
kernel already knows the inode).

So I'd expect the code to be somewhere in fuse_lib_lookup(). The only
candidate in there is the call to lookup_path(), but this in turn calls
fuse_lib_getattr() - which must not overwrite the inode.

In other words, I am not sure how exactly use_ino is supposed to work,
nor how it is actually working at the moment.

> Perhaps more perplexing is that when the inode is 0 (with use_ino) running
> in debug mode shows that readdir returns successfully with the same amount
> of data that I see when leave libfuse to generate the inodes

Well, that is expected. The data returned to the kernel has always the
same length, no matter if the inode is defined by your filesystem or by
the high-level API internally.

Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: setting inodes in the highlevel API

Ambrose Feinstein
I'm pretty sure d_type makes it out to userspace and programs may act
on it without triggering a stat, eg 'find . -name core' doesn't need
to recurse into a DT_REG entry.  It's just not used very much, since
you still need fallback code to handle DT_UNKNOWN.

As for d_ino, I think what you're seeing is glibc behavior where d_ino
== 0 from getdents(2) causes the record to be discarded and not
returned from readdir(3).  I think this is some very old hack for
representing deleted files or handling concurrent modifications.  Just
avoid using 0 as an inode number.


On Thu, Dec 15, 2016 at 12:31 PM, Antonio SJ Musumeci
<[hidden email]> wrote:

> The little documentation I've found[0] says that the inode doesn't need to
> be unique which implies that the value may not really matter to the system.
>
> I remember a while back noticing that setting dirent.d_type only mattered if
> the followup stat failed. I would suspect that is also true for the d_ino.
> Let me test that.
>
> [0] fuse.h reads
> /**
> * Honor the st_ino field in the functions getattr() and
> * fill_dir(). This value is used to fill in the st_ino field
> * in the stat(2), lstat(2), fstat(2) functions and the d_ino
> * field in the readdir(2) function. The filesystem does not
> * have to guarantee uniqueness, however some applications
> * rely on this value being unique for the whole filesystem.
> */
>
> On Thu, Dec 15, 2016 at 11:47 AM, Nikolaus Rath <[hidden email]> wrote:
>>
>> On Dec 13 2016, Antonio SJ Musumeci
>> <[hidden email]> wrote:
>> > I take that back.
>> >
>> > When use_ino isn't used fill_dir sets stbuf.st_ino = FUSE_UNKNOWN_INO ==
>> > 0xFFFFFFFF.
>> >
>> > I'm not able to find that special value used anywhere in the kernel code
>> > nor outside the highlevel libfuse code for anything interesting. Appears
>> > that so long as the value is !zero getattr overwrites it. Speaking of
>> > which
>> > where does that happen?
>>
>> I think overwriting the inode used by the high-level API must happen
>> only if getattr() is called as a result of a low-level lookup() request,
>> but not for a low-level getattr() request (because in that case the
>> kernel already knows the inode).
>>
>> So I'd expect the code to be somewhere in fuse_lib_lookup(). The only
>> candidate in there is the call to lookup_path(), but this in turn calls
>> fuse_lib_getattr() - which must not overwrite the inode.
>>
>> In other words, I am not sure how exactly use_ino is supposed to work,
>> nor how it is actually working at the moment.
>>
>> > Perhaps more perplexing is that when the inode is 0 (with use_ino)
>> > running
>> > in debug mode shows that readdir returns successfully with the same
>> > amount
>> > of data that I see when leave libfuse to generate the inodes
>>
>> Well, that is expected. The data returned to the kernel has always the
>> same length, no matter if the inode is defined by your filesystem or by
>> the high-level API internally.
>>
>> Best,
>> -Nikolaus
>>
>> --
>> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
>> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>>
>>              »Time flies like an arrow, fruit flies like a Banana.«
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> --
>> fuse-devel mailing list
>> To unsubscribe or subscribe, visit
>> https://lists.sourceforge.net/lists/listinfo/fuse-devel
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> --
> fuse-devel mailing list
> To unsubscribe or subscribe, visit
> https://lists.sourceforge.net/lists/listinfo/fuse-devel
>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel