Problem if "open()" is too fast

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Problem if "open()" is too fast

Haskins, Gregory
Hi All,
  I dont know if this is known or not.  I tried searching to no avail, so I apologize if this has already been covered.

I am using fuse 2.3.0 on linux 2.4 in multithreaded mode.

I had written a little app that was very similar to the example provided with fuse and it worked great.  Then I tried to write a fuse-app that was a little more advanced and was maintaining all of its state inside STL maps/lists internal to the fuse-app.  But when I would "cat" a file from my filesystem, it would come back as empty (even though it actually had data in it).

So I placed a debugger in my code and suddenly things were working!  Take away the debugger, and sure enough it would break again.  Im not really sure what this means, but by using the "-d" flag I finally figured it out that if the OPEN function returns "too fast", fuse follows up the OPEN with an immediate RELEASE.  But if I put a "sleep(1)" in my open, the OPEN is followed by GETATTR, etc, and then the RELEASE.  Based on this, it led me to believe that the problem is with fuse itself, not my app (but I cannot be sure).

So as a temporary stop-gap, all of my opens have a 1 second sleep.  This is somewhat awkward for the user to have this delay, so I thought I would poke around out there and see if there is a solution.  Am I just doing something wrong?  Is this a known issue with fuse?  Is there some info I can provide to someone out there to help diagnose this problem?

Thanks a bunch!
-Greg

winmail.dat (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem if "open()" is too fast

Miklos Szeredi
>   I dont know if this is known or not.  I tried searching to no
> avail, so I apologize if this has already been covered.
>
> I am using fuse 2.3.0 on linux 2.4 in multithreaded mode.
>
> I had written a little app that was very similar to the example
> provided with fuse and it worked great.  Then I tried to write a
> fuse-app that was a little more advanced and was maintaining all of
> its state inside STL maps/lists internal to the fuse-app.  But when
> I would "cat" a file from my filesystem, it would come back as empty
> (even though it actually had data in it).
>
> So I placed a debugger in my code and suddenly things were working!
> Take away the debugger, and sure enough it would break again.  Im
> not really sure what this means, but by using the "-d" flag I
> finally figured it out that if the OPEN function returns "too fast",
> fuse follows up the OPEN with an immediate RELEASE.  But if I put a
> "sleep(1)" in my open, the OPEN is followed by GETATTR, etc, and
> then the RELEASE.  Based on this, it led me to believe that the
> problem is with fuse itself, not my app (but I cannot be sure).

Maybe you are returning different values for st_size in the initial
getattr() call and the one after open.  That could explain, why a
sleep(1) makes a difference.

> So as a temporary stop-gap, all of my opens have a 1 second sleep.
> This is somewhat awkward for the user to have this delay, so I
> thought I would poke around out there and see if there is a
> solution.  Am I just doing something wrong?  Is this a known issue
> with fuse?  Is there some info I can provide to someone out there to
> help diagnose this problem?

Try printing out stbuf->st_size at the end of your getattr() method.

Miklos


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Problem if "open()" is too fast

Haskins, Gregory
In reply to this post by Haskins, Gregory
 
>
> Maybe you are returning different values for st_size in the initial
> getattr() call and the one after open.  That could explain, why a
> sleep(1) makes a difference.
>

I actually am doing exactly that in the code, but I believe the problem
occurs before I get a chance to actually report the difference to FUSE.


I am basically making a "user-mode /proc" filesystem.  The inodes in my
filesystem are virtual and therefore I report the st_size as "0" when
GETATTR scans through during READDIR and on any closed files because I
do not know the size of the data apriori.  

Once the file is actually opened, however, I make a stream-connection to
the application who owns the node and transfer all the data over locally
to the open-file-object in the fuse-app.  So the open-file-object does
in fact know its size unlike the inode object.  

In my original implementation I used to return 0 for all nodes during
GETATTR.  When I first ran into this problem I was thinking that maybe I
was confusing FUSE into thinking it doesn't need to read because it
knows the file is 0.  So I changed my implementation as indicated above
(i.e. open objects know their real size).

However, what I see is the following when I cat a file (via "-d")

Without the sleep:

GETATTR (st_size=0)
OPEN
RELEASE

With the sleep:

GETATTR (st_size=0)
OPEN (sleep(1))
GETATTR (st_size=N)
READ
RELEASE

Note that all of the fuse queries are indicating success codes.  The
difference is that I never get the GETATTR/READ after the open if open
returns quickly.  If I have a breakpoint or a sleep in the open, it
works.  Otherwise, it seems content to request a release immediately
after the open.

>
> Try printing out stbuf->st_size at the end of your getattr() method.

I can try, but the one before the open I know is always "0" and the one
after the open doesn't get invoked if open is too fast.  Would you still
like me to try?

Thanks a bunch for the quick replies.  Fuse is a really cool idea.
-Greg


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. <a href="http://ads.osdn.com/?ad_idt77&alloc_id492&op=click">http://ads.osdn.com/?ad_idt77&alloc_id492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem if "open()" is too fast

Miklos Szeredi
> >
> > Maybe you are returning different values for st_size in the initial
> > getattr() call and the one after open.  That could explain, why a
> > sleep(1) makes a difference.
> >
>
> I actually am doing exactly that in the code, but I believe the problem
> occurs before I get a chance to actually report the difference to FUSE.
>
>
> I am basically making a "user-mode /proc" filesystem.  The inodes in my
> filesystem are virtual and therefore I report the st_size as "0" when
> GETATTR scans through during READDIR and on any closed files because I
> do not know the size of the data apriori.  

In this case you should use the '-odirect_io' mount option.  Then you
can set st_size to zero, and still get the read() requests.

Otherwise the kernel will assume that the file is zero size, and will
not issue the read() request.

Miklos


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Problem if "open()" is too fast

Haskins, Gregory
In reply to this post by Haskins, Gregory
> > I am basically making a "user-mode /proc" filesystem.  The inodes in

> > my filesystem are virtual and therefore I report the st_size as "0"
> > when GETATTR scans through during READDIR and on any closed files
> > because I do not know the size of the data apriori.
>
> In this case you should use the '-odirect_io' mount option.  
> Then you can set st_size to zero, and still get the read() requests.
>
> Otherwise the kernel will assume that the file is zero size,
> and will not issue the read() request.

Yep, you nailed it!  Its working like a champ now without the sleep.
Thanks!

Just purely out of curiosity:  Any ideas about why the sleep(1) made it
work?  Is it that the kernel/vfs went into some timeout condition
because I took so long and then it re-issued the second GETATTR (which I
just happen to now return with a non-zero size)?

I.e. Is the second GETATTR pathological normally or does that happen
after every open all the time?  Was the fact that the second query was
non-zero the part that actually enabled the read to take place?

-Greg



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. <a href="http://ads.osdn.com/?ad_idt77&alloc_id492&op=click">http://ads.osdn.com/?ad_idt77&alloc_id492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem if "open()" is too fast

Miklos Szeredi
> Just purely out of curiosity:  Any ideas about why the sleep(1) made it
> work?  Is it that the kernel/vfs went into some timeout condition
> because I took so long and then it re-issued the second GETATTR (which I
> just happen to now return with a non-zero size)?
>
> I.e. Is the second GETATTR pathological normally or does that happen
> after every open all the time?

It only happens if the attribute cache has expired (1s) and there was
a request for the attributes (e.g. fstat()).  You satisfied the first
condition with the sleep(1), and the second condition just happens to
be true for 'cat'.

> Was the fact that the second query was non-zero the part that
> actually enabled the read to take place?

Yes.

Miklos



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Loading...