Quantcast

llfuse lookup operation issue

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

llfuse lookup operation issue

Cameron Simpson
Hi,

I'm really trying to shut up some messages that emit from llfuse when a lookup
fails. My lookup stuff has this:

  try:
    E = P[name]
  except KeyError:
    raise FuseOSError(errno.ENOENT)

I get this in the window where the daemon is running:

  File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 89, in handle
    return method(self, *a, **kw)
  File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 840, in lookup
    raise FuseError(errno.ENOENT)

whenever anything tries to stat something that doesn't exist. So I look at the
lookup operation here:

  https://pythonhosted.org/llfuse/operations.html#llfuse.Operations.lookup

and it says:

  If there is no such entry, the method should either return an EntryAttributes
  instance with negative st_ino value (in which case the negative lookup will
  be cached as specified by entry_timeout), or it should raise FUSEError with
  an errno of errno.ENOENT (in this case the negative result will not be
  cached).

So I tried this:

  try:
    E = P[name]
  except KeyError:
    EA = llfuse.EntryAttributes()
    EA.st_ino = -1
    EA.entry_timeout = 1.0
    return EA

and this happens:

    File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 89, in handle
      return method(self, *a, **kw)
    File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 836, in lookup
      EA.st_ino = -1
    File "src/misc.pxi", line 376, in llfuse.EntryAttributes.st_ino.__set__ (src/llfuse.c:29166)
  2017-01-30 10:33:36,418: vt: BANG1: e=<class 'OverflowError'> lookup: can't convert negative value to fuse_ino_t
  2017-01-30 10:33:36,419: vt: EXCEPTION from .lookup(*(1, b'cs', <llfuse.RequestContext object at 0x10e72bd10>),**{}): lookup: can't convert negative value to fuse_ino_t

which seems in conflict with the documentation. (The "BANG1" and "EXCEPTION"
messages come from my operation wrapper; I'm concerned that the OverflowError
is happening at all - you can see my code doesn't get to return EA at all.)

Am I doing something wrong here?

Cheers,
Cameron Simpson <[hidden email]>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Michael Theall-2
Hi Cameron,

I haven't use the python interfaces, but perhaps that documentation is simply incorrect. The C libfuse documentation states that providing st_ino with the value of '0' will cause the negative lookup to be cached. I hope that helps.

struct fuse_entry_param {
/** Unique inode number
*
* In lookup, zero means negative entry (from version 2.5)
* Returning ENOENT also means negative entry, but by setting zero
* ino the kernel may cache negative entries for entry_timeout
* seconds.
*/
fuse_ino_t ino;

Regards,
Michael Theall

On Sun, Jan 29, 2017 at 5:59 PM Cameron Simpson <[hidden email]> wrote:
Hi,

I'm really trying to shut up some messages that emit from llfuse when a lookup
fails. My lookup stuff has this:

  try:
    E = P[name]
  except KeyError:
    raise FuseOSError(errno.ENOENT)

I get this in the window where the daemon is running:

  File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 89, in handle
    return method(self, *a, **kw)
  File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 840, in lookup
    raise FuseError(errno.ENOENT)

whenever anything tries to stat something that doesn't exist. So I look at the
lookup operation here:

  https://pythonhosted.org/llfuse/operations.html#llfuse.Operations.lookup

and it says:

  If there is no such entry, the method should either return an EntryAttributes
  instance with negative st_ino value (in which case the negative lookup will
  be cached as specified by entry_timeout), or it should raise FUSEError with
  an errno of errno.ENOENT (in this case the negative result will not be
  cached).

So I tried this:

  try:
    E = P[name]
  except KeyError:
    EA = llfuse.EntryAttributes()
    EA.st_ino = -1
    EA.entry_timeout = 1.0
    return EA

and this happens:

    File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 89, in handle
      return method(self, *a, **kw)
    File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 836, in lookup
      EA.st_ino = -1
    File "src/misc.pxi", line 376, in llfuse.EntryAttributes.st_ino.__set__ (src/llfuse.c:29166)
  2017-01-30 10:33:36,418: vt: BANG1: e=<class 'OverflowError'> lookup: can't convert negative value to fuse_ino_t
  2017-01-30 10:33:36,419: vt: EXCEPTION from .lookup(*(1, b'cs', <llfuse.RequestContext object at 0x10e72bd10>),**{}): lookup: can't convert negative value to fuse_ino_t

which seems in conflict with the documentation. (The "BANG1" and "EXCEPTION"
messages come from my operation wrapper; I'm concerned that the OverflowError
is happening at all - you can see my code doesn't get to return EA at all.)

Am I doing something wrong here?

Cheers,
Cameron Simpson <[hidden email]>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Nikolaus Rath
In reply to this post by Cameron Simpson
On Jan 30 2017, Cameron Simpson <[hidden email]> wrote:

> Hi,
>
> I'm really trying to shut up some messages that emit from llfuse when a lookup
> fails. My lookup stuff has this:
>
>   try:
>     E = P[name]
>   except KeyError:
>     raise FuseOSError(errno.ENOENT)
>
> I get this in the window where the daemon is running:
>
>   File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 89, in handle
>     return method(self, *a, **kw)
>   File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 840, in lookup
>     raise FuseError(errno.ENOENT)
>
> whenever anything tries to stat something that doesn't exist.

(I assume you have defined FuseOsError == FUSEError)

Do you have the same problem with the example filesystem? Because that
should work and as far as I can tell does work. See eg.

https://bitbucket.org/nikratio/python-llfuse/src/tip/examples/tmpfs.py#tmpfs.py-140

> So I look at the
> lookup operation here:
>
>   https://pythonhosted.org/llfuse/operations.html#llfuse.Operations.lookup
>
> and it says:
>
>   If there is no such entry, the method should either return an EntryAttributes
>   instance with negative st_ino value (in which case the negative lookup will
>   be cached as specified by entry_timeout), or it should raise FUSEError with
>   an errno of errno.ENOENT (in this case the negative result will not be
>   cached).

Ops, this is a bug (either in documentation or implementation). Could
you file an issue at
https://bitbucket.org/nikratio/python-llfuse/issues?



Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Cameron Simpson
On 30Jan2017 16:56, Nikolaus Rath <[hidden email]> wrote:

>On Jan 30 2017, Cameron Simpson <[hidden email]> wrote:
>> I'm really trying to shut up some messages that emit from llfuse when a
>> lookup fails. My lookup stuff has this:
>>
>>   try:
>>     E = P[name]
>>   except KeyError:
>>     raise FuseOSError(errno.ENOENT)
>>
>> I get this in the window where the daemon is running:
>>
>>   File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 89, in handle
>>     return method(self, *a, **kw)
>>   File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 840, in lookup
>>     raise FuseError(errno.ENOENT)
>>
>> whenever anything tries to stat something that doesn't exist.
>
>(I assume you have defined FuseOsError == FUSEError)

Yes, sorry. Left over from porting from fuse3, which didn't work very well in
the long run. llfuse is pretty good and has cleaner inerfaces as well! From
here:

  FUSE_CLASS = 'llfuse'

  if FUSE_CLASS == 'llfuse':
    import llfuse
    from llfuse import FUSEError as FuseOSError
    FuseOSError = llfuse.FUSEError
  elif FUSE_CLASS == 'fuse3':
    # my slightly hacked python-fuse with crude python 3 porting hacks
    from fuse3 import FUSEOSError

>Do you have the same problem with the example filesystem? Because that
>should work and as far as I can tell does work. See eg.
>https://bitbucket.org/nikratio/python-llfuse/src/tip/examples/tmpfs.py#tmpfs.py-140

That code raises ENOENT:

  try:
    inode = self.get_row("SELECT * FROM contents WHERE name=? AND parent_inode=?",
                         (name, inode_p))['inode']
  except NoSuchRowError:
    raise(llfuse.FUSEError(errno.ENOENT))

That's what I'm currently doing. I was hoping returning an EntryAttributes with
st_ino = -1 might make for less noise. I'm undecided as to whether I want the
"cache the negative lookup" part yet.

>> So I look at the
>> lookup operation here:
>>   https://pythonhosted.org/llfuse/operations.html#llfuse.Operations.lookup
>>
>> and it says:
>>   If there is no such entry, the method should either return an
>>   EntryAttributes
>>   instance with negative st_ino value (in which case the negative lookup will
>>   be cached as specified by entry_timeout), or it should raise FUSEError with
>>   an errno of errno.ENOENT (in this case the negative result will not be
>>   cached).
>
>Ops, this is a bug (either in documentation or implementation). Could
>you file an issue at
>https://bitbucket.org/nikratio/python-llfuse/issues?

Done:

  https://bitbucket.org/nikratio/python-llfuse/issues/106/llfuse-lookup-operation-does-not-support

I'll exercise the st_ino=0 suggestion from Michael Theall soon.

Thanks,
Cameron Simpson <[hidden email]>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Cameron Simpson
In reply to this post by Michael Theall-2
On 30Jan2017 15:05, Michael Theall <[hidden email]> wrote:
>I haven't use the python interfaces, but perhaps that documentation is
>simply incorrect. The C libfuse documentation states that providing st_ino
>with the value of '0' will cause the negative lookup to be cached. I hope
>that helps.
[...details snipped...]

Thank you, I'll try that this evening with luck.

I've made a bug for llfuse here:

  https://bitbucket.org/nikratio/python-llfuse/issues/106/llfuse-lookup-operation-does-not-support

to track this.

Cheers,
Cameron Simpson <[hidden email]>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Cameron Simpson
On 31Jan2017 15:14, Cameron Simpson <[hidden email]> wrote:
>On 30Jan2017 15:05, Michael Theall <[hidden email]> wrote:
>>I haven't use the python interfaces, but perhaps that documentation is
>>simply incorrect. The C libfuse documentation states that providing st_ino
>>with the value of '0' will cause the negative lookup to be cached. I hope
>>that helps.
>[...details snipped...]
>
>Thank you, I'll try that this evening with luck.

Seems that st_ino=0 works. Thanks. I've updated the issue to mention this.

Cheers,
Cameron Simpson <[hidden email]>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Nikolaus Rath
In reply to this post by Cameron Simpson
On Jan 31 2017, Cameron Simpson <[hidden email]> wrote:

> On 30Jan2017 16:56, Nikolaus Rath <[hidden email]> wrote:
>>On Jan 30 2017, Cameron Simpson <[hidden email]> wrote:
>>> I'm really trying to shut up some messages that emit from llfuse when a
>>> lookup fails. My lookup stuff has this:
>>>
>>>   try:
>>>     E = P[name]
>>>   except KeyError:
>>>     raise FuseOSError(errno.ENOENT)
>>>
>>> I get this in the window where the daemon is running:
>>>
>>>   File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 89, in handle
>>>     return method(self, *a, **kw)
>>>   File "/Users/cameron/hg/css-venti/cs/venti/vtfuse.py", line 840, in lookup
>>>     raise FuseError(errno.ENOENT)
>>>
>>> whenever anything tries to stat something that doesn't exist.
>>
>>(I assume you have defined FuseOsError == FUSEError)
>
> Yes, sorry. Left over from porting from fuse3, which didn't work very well in
> the long run. llfuse is pretty good and has cleaner inerfaces as well! From
> here:
>
>   FUSE_CLASS = 'llfuse'
>
>   if FUSE_CLASS == 'llfuse':
>     import llfuse
>     from llfuse import FUSEError as FuseOSError
>     FuseOSError = llfuse.FUSEError

The last two lines do the same thing, you need only one of them.

>   elif FUSE_CLASS == 'fuse3':
>     # my slightly hacked python-fuse with crude python 3 porting hacks
>     from fuse3 import FUSEOSError
>
>>Do you have the same problem with the example filesystem? Because that
>>should work and as far as I can tell does work. See eg.
>>https://bitbucket.org/nikratio/python-llfuse/src/tip/examples/tmpfs.py#tmpfs.py-140
>
> That code raises ENOENT:

Yes, just like your code. That's why I am wondering if it gives you the
same problem.

>
>   try:
>     inode = self.get_row("SELECT * FROM contents WHERE name=? AND parent_inode=?",
>                          (name, inode_p))['inode']
>   except NoSuchRowError:
>     raise(llfuse.FUSEError(errno.ENOENT))
>
> That's what I'm currently doing. I was hoping returning an EntryAttributes with
> st_ino = -1 might make for less noise.

What do you mean with "less noise"? Are you saying you are getting
exception tracebacks printed when you run the example?


Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Cameron Simpson
On 31Jan2017 20:22, Nikolaus Rath <[hidden email]> wrote:
>On Jan 31 2017, Cameron Simpson <[hidden email]> wrote:
>>     from llfuse import FUSEError as FuseOSError
>>     FuseOSError = llfuse.FUSEError
>
>The last two lines do the same thing, you need only one of them.

Yeah, already fixed :-) Only saw it after I'd recited them here. Cruft from the
port.

>>   elif FUSE_CLASS == 'fuse3':
>>     # my slightly hacked python-fuse with crude python 3 porting hacks
>>     from fuse3 import FUSEOSError
>>
>>>Do you have the same problem with the example filesystem? Because that
>>>should work and as far as I can tell does work. See eg.
>>>https://bitbucket.org/nikratio/python-llfuse/src/tip/examples/tmpfs.py#tmpfs.py-140
>>
>> That code raises ENOENT:
>
>Yes, just like your code. That's why I am wondering if it gives you the
>same problem.

No, but raising ENOENT produces a traceback on stderr. Unless I've got
something in my own code doing that. So I was trying the alternative return as
an experiment, and tripped over the st_ino issue. I'm running with st_ino=0 for
now, as noted in the ticket, and i might keep it for the negative lookup
caching; undecided on that so far.

The ticket itself is about the doco saying st_ino=-1 and the exception that
causes.

>>   try:
>>     inode = self.get_row("SELECT * FROM contents WHERE name=? AND parent_inode=?",
>>                          (name, inode_p))['inode']
>>   except NoSuchRowError:
>>     raise(llfuse.FUSEError(errno.ENOENT))
>>
>> That's what I'm currently doing. I was hoping returning an EntryAttributes with
>> st_ino = -1 might make for less noise.
>
>What do you mean with "less noise"? Are you saying you are getting
>exception tracebacks printed when you run the example?

I'll have to try the same filesystem to see. Either llfuse is printing some
traceback when lookup raises ENOENT, or something in my own code is, which I
don't think is the case. All my operations are presently wrapped in this
decorator for ease of debugging:

  def handler(method):
    ''' Decorator for FUSE handlers.
        Prefixes exceptions with the method name, associated with the
        Store, prevents anything other than a FUSEOSError being raised.
    '''
    def handle(self, *a, **kw):
      try:
        with Pfx(method.__name__):
          with self._vt_core.S:
            return method(self, *a, **kw)
      except FuseOSError:
        raise
      except MissingHashcodeError as e:
        raise FuseOSError(errno.EIO) from e
      except OSError as e:
        raise FuseOSError(e.errno) from e
      except Exception as e:
        error("BANG1: e=%s %s", type(e), e)
        exception("EXCEPTION from .%s(*%r,**%r): %s", method.__name__, a, kw, e)
        raise FuseOSError(errno.EINVAL) from e
      except:
        error("UNCAUGHT EXCEPTION")
        raise RuntimeError("UNCAUGHT EXCEPTION")
    return handle

and the first thing that does is simply reraise FuseError directly before
specially handling various other disasters. So lookup itself just looks like
this:

  @handler
  def lookup(self, parent_inode, name_b, ctx):
    ..............

I don't believe there's any scope for my stuff producing a partial traceback
when lookup raises ENOENT.

Cheers,
Cameron Simpson <[hidden email]>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Nikolaus Rath
On Feb 01 2017, Cameron Simpson <[hidden email]> wrote:

>>>>Do you have the same problem with the example filesystem? Because that
>>>>should work and as far as I can tell does work. See eg.
>>>>https://bitbucket.org/nikratio/python-llfuse/src/tip/examples/tmpfs.py#tmpfs.py-140
>>>
>>> That code raises ENOENT:
>>
>>Yes, just like your code. That's why I am wondering if it gives you the
>>same problem.
>
> No, but raising ENOENT produces a traceback on stderr. Unless I've got
> something in my own code doing that.
[...]

>>>   try:
>>>     inode = self.get_row("SELECT * FROM contents WHERE name=? AND parent_inode=?",
>>>                          (name, inode_p))['inode']
>>>   except NoSuchRowError:
>>>     raise(llfuse.FUSEError(errno.ENOENT))
>>>
>>> That's what I'm currently doing. I was hoping returning an EntryAttributes with
>>> st_ino = -1 might make for less noise.
>>
>>What do you mean with "less noise"? Are you saying you are getting
>>exception tracebacks printed when you run the example?
>
> I'll have to try the same filesystem to see. Either llfuse is printing some
> traceback when lookup raises ENOENT, or something in my own code is, which I
> don't think is the case.

I am pretty sure that llfuse doesn't have any such code. So I'd much
appreciate if you could try to reproduce this message with the example
filesystem.

Best,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: llfuse lookup operation issue

Cameron Simpson
On 02Feb2017 15:10, Nikolaus Rath <[hidden email]> wrote:

>On Feb 01 2017, Cameron Simpson <[hidden email]> wrote:
>>>> That's what I'm currently doing. I was hoping returning an EntryAttributes
>>>> with
>>>> st_ino = -1 might make for less noise.
>>>
>>>What do you mean with "less noise"? Are you saying you are getting
>>>exception tracebacks printed when you run the example?
>>
>> I'll have to try the same filesystem to see. Either llfuse is printing some
>> traceback when lookup raises ENOENT, or something in my own code is, which I
>> don't think is the case.
>
>I am pretty sure that llfuse doesn't have any such code. So I'd much
>appreciate if you could try to reproduce this message with the example
>filesystem.

I'm giving that a spin at the moment. As you suggest, no traceback.

Clearly the traceback comes from my end somehow. I'll dig more.

Thanks for your help,
Cameron Simpson <[hidden email]>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Loading...