[python-fuse] Nanosecond file times?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[python-fuse] Nanosecond file times?

Nikolaus Rath
Hi Csaba,

I understand that in principle python fuse supports micro- and
nanosecond file access/modification times. Could you explain to me how I
can implement this in practice?

What kind of data structure do the (python) utimens and getrattr
functions receive and have to return?


Thanks,

   -Nikolaus

--
 »Time flies like an arrow, fruit flies like a Banana.«

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com 
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: [python-fuse] Nanosecond file times?

Csaba Henk
Hi Nick,

I could have answered this earlier (given you sent me a courtesy copy)
but if you post to the list, I'd rather answer there... but it's soo
much time while a posting appears on gmane :/ Anyways,

On 2009-05-27, Nikolaus Rath <[hidden email]> wrote:
> Hi Csaba,
>
> I understand that in principle python fuse supports micro- and
> nanosecond file access/modification times. Could you explain to me how I
> can implement this in practice?
>
> What kind of data structure do the (python) utimens and getrattr
> functions receive and have to return?

in the C code:

---------------------------------------------------
static int
utimens_func(const char *path, const struct timespec ts[2])
{
        PROLOGUE(
          PyObject_CallFunction(utimens_cb, "siiii", path,
                                ts[0].tv_sec, ts[0].tv_nsec,
                                ts[1].tv_sec, ts[1].tv_nsec)
        )

        EPILOGUE
}
---------------------------------------------------

in the Python code:

---------------------------------------------------
class Timespec(FuseStruct):
    """
    Cf. struct timespec in time.h:
    http://www.opengroup.org/onlinepubs/009695399/basedefs/time.h.html
    """

    def __init__(self, **kw):

        self.tv_sec  = None
        self.tv_nsec = None

        FuseStruct.__init__(self, **kw)
[...]
    def lowwrap(self, fname):
[...]
        elif fname == 'utimens':
            def wrap(path, acc_sec, acc_nsec, mod_sec, mod_nsec):
                ts_acc = Timespec(tv_sec = acc_sec, tv_nsec = acc_nsec)
                ts_mod = Timespec(tv_sec = mod_sec, tv_nsec = mod_nsec)
                return fun(path, ts_acc, ts_mod)
---------------------------------------------------

ie. your callback gets the path, and two Timespec instances, for access and
modification times, which feature the tv_sec and tv_nsec attributes. You don't
have to return anything (unless you want an errory return), just perform a side
effect.

For getattr, you get the path, and have to return an object which has
the necessary stat fields (and may have some optional ones) as
attributes. C code:

---------------------------------------------------
#define FETCH_STAT_DATA()                                               \
        fetchattr(st, st_mode);                                         \
        fetchattr(st, st_ino);                                          \
        fetchattr(st, st_dev);                                          \
        fetchattr(st, st_nlink);                                        \
        fetchattr(st, st_uid);                                          \
        fetchattr(st, st_gid);                                          \
        fetchattr(st, st_size);                                         \
        fetchattr(st, st_atime);                                        \
        fetchattr(st, st_mtime);                                        \
        fetchattr(st, st_ctime);                                        \
                                                                        \
        /*                                                              \
         * Following fields are not necessarily available on all        \
         * platforms (were "all" stands for "POSIX-like"). Therefore    \
         * we should have some #ifdef-s around... However, they _are_   \
         * available on those platforms where FUSE has a chance to      \
         * run now and in the foreseeable future, and we don't use      \
         * autotools so we just dare to throw these in as is.           \
         */                                                             \
                                                                        \
        fetchattr_soft(st, st_rdev);                                    \
        fetchattr_soft_d(st, st_blksize, 4096);                         \
        fetchattr_soft_d(st, st_blocks, (st->st_size + 511)/512)
---------------------------------------------------

fetchattr'd attributes are the obligate ones, fechattr_softed ones are
optional.

Do you see any ambiguity here? Or you just have overlooked the "lowwrap"
stuff (which is there to make the interface a bit more humane
friendly...)?

Csaba


------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com 
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: [python-fuse] Nanosecond file times?

Nikolaus Rath
Csaba Henk <[hidden email]> writes:

> On 2009-05-27, Nikolaus Rath <[hidden email]> wrote:
>> Hi Csaba,
>>
>> I understand that in principle python fuse supports micro- and
>> nanosecond file access/modification times. Could you explain to me how I
>> can implement this in practice?
>>
>> What kind of data structure do the (python) utimens and getrattr
>> functions receive and have to return?
>
> in the C code:
> [...]
>
> ie. your callback gets the path, and two Timespec instances, for access and
> modification times, which feature the tv_sec and tv_nsec attributes. You don't
> have to return anything (unless you want an errory return), just perform a side
> effect.

I see, thanks!

> For getattr, you get the path, and have to return an object which has
> the necessary stat fields (and may have some optional ones) as
> attributes. C code:
>
> ---------------------------------------------------
> #define FETCH_STAT_DATA()                                               \
>         fetchattr(st, st_mode);                                         \
>         fetchattr(st, st_ino);                                          \
>         fetchattr(st, st_dev);                                          \
>         fetchattr(st, st_nlink);                                        \
>         fetchattr(st, st_uid);                                          \
>         fetchattr(st, st_gid);                                          \
>         fetchattr(st, st_size);                                         \
>         fetchattr(st, st_atime);                                        \
>         fetchattr(st, st_mtime);                                        \
>         fetchattr(st, st_ctime);                                        \
>                                                                         \
>         /*                                                              \
>          * Following fields are not necessarily available on all        \
>          * platforms (were "all" stands for "POSIX-like"). Therefore    \
>          * we should have some #ifdef-s around... However, they _are_   \
>          * available on those platforms where FUSE has a chance to      \
>          * run now and in the foreseeable future, and we don't use      \
>          * autotools so we just dare to throw these in as is.           \
>          */                                                             \
>                                                                         \
>         fetchattr_soft(st, st_rdev);                                    \
>         fetchattr_soft_d(st, st_blksize, 4096);                         \
>         fetchattr_soft_d(st, st_blocks, (st->st_size + 511)/512)
> ---------------------------------------------------
>
> fetchattr'd attributes are the obligate ones, fechattr_softed ones are
> optional.

Yes, but aren't st_{amt}time of type time_t and thus seconds since 1970?
How do I fit a higher resolution in there?


Best,

   -Nikolaus

--
 »Time flies like an arrow, fruit flies like a Banana.«

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com 
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: [python-fuse] Nanosecond file times?

Bugzilla from john.haxby@oracle.com
Nikolaus Rath wrote:
> Yes, but aren't st_{amt}time of type time_t and thus seconds since 1970?
> How do I fit a higher resolution in there?
>  

Here (at least) st_{amc}time are defined as st_{amc}tim.tv_sec;
st_{amc}tim.tv_nsec are the nanonsecond times.

jch

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com 
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: [python-fuse] Nanosecond file times?

Csaba Henk
In reply to this post by Nikolaus Rath
On 2009-06-01, Nikolaus Rath <[hidden email]> wrote:

> Csaba Henk <[hidden email]> writes:
>> For getattr, you get the path, and have to return an object which has
>> the necessary stat fields (and may have some optional ones) as
>> attributes. C code:
>>
>> ---------------------------------------------------
>> #define FETCH_STAT_DATA()                                               \
>>         fetchattr(st, st_mode);                                         \
>>         fetchattr(st, st_ino);                                          \
>>         fetchattr(st, st_dev);                                          \
>>         fetchattr(st, st_nlink);                                        \
>>         fetchattr(st, st_uid);                                          \
>>         fetchattr(st, st_gid);                                          \
>>         fetchattr(st, st_size);                                         \
>>         fetchattr(st, st_atime);                                        \
>>         fetchattr(st, st_mtime);                                        \
>>         fetchattr(st, st_ctime);                                        \
>>                                                                         \
>>         /*                                                              \
>>          * Following fields are not necessarily available on all        \
>>          * platforms (were "all" stands for "POSIX-like"). Therefore    \
>>          * we should have some #ifdef-s around... However, they _are_   \
>>          * available on those platforms where FUSE has a chance to      \
>>          * run now and in the foreseeable future, and we don't use      \
>>          * autotools so we just dare to throw these in as is.           \
>>          */                                                             \
>>                                                                         \
>>         fetchattr_soft(st, st_rdev);                                    \
>>         fetchattr_soft_d(st, st_blksize, 4096);                         \
>>         fetchattr_soft_d(st, st_blocks, (st->st_size + 511)/512)
>> ---------------------------------------------------
>>
>> fetchattr'd attributes are the obligate ones, fechattr_softed ones are
>> optional.
>
> Yes, but aren't st_{amt}time of type time_t and thus seconds since 1970?
> How do I fit a higher resolution in there?

See SUSv3:

http://www.opengroup.org/onlinepubs/009695399/basedefs/sys/stat.h.html

vs SUSv4 aka POSIX.1-2008 (this is not mentioned yet in Wikipedia! but
cf. standards(7) [assumed you have a recent man-pages package
installed]):

http://www.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html

In SUSv3, it is as you say, the st_*time fields are of time_t, letting
thus only a second grained resolution. In SUSv4 though, there are no
st_*time fields, but there are st_*tim fields, which are struct
timespec; moreover

"""
For compatibility with earlier versions of this standard, the st_atime
macro shall be defined with the value st_atim.tv_sec. Similarly,
st_ctime and st_mtime shall be defined as macros with the values
st_ctim.tv_sec and st_mtim.tv_sec, respectively.
"""

So yes, probably it would be time to adopt SUSv4 for the python
binding. This can happen sooner if someone yells that it's a real itch
for him/her. Is it a real itch for you, or are you just wondering?

Csaba


------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com 
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: [python-fuse] Nanosecond file times?

Nikolaus Rath
Csaba Henk <[hidden email]> writes:

> On 2009-06-01, Nikolaus Rath <[hidden email]> wrote:
>> Csaba Henk <[hidden email]> writes:
>>> For getattr, you get the path, and have to return an object which has
>>> the necessary stat fields (and may have some optional ones) as
>>> attributes. C code:
>>
>> Yes, but aren't st_{amt}time of type time_t and thus seconds since 1970?
>> How do I fit a higher resolution in there?
>
> In SUSv3, it is as you say, the st_*time fields are of time_t, letting
> thus only a second grained resolution. In SUSv4 though, there are no
> st_*time fields, but there are st_*tim fields, which are struct
> timespec; moreover
>
> So yes, probably it would be time to adopt SUSv4 for the python
> binding. This can happen sooner if someone yells that it's a real itch
> for him/her. Is it a real itch for you, or are you just wondering?

It's an itch for me, but nothing I can't work around.

I am currently writing unit tests for my filesystem and I wanted to
check if {amc}times are updated correctly by the different handlers.
Without proper time resolution, I have to insert sleep(1)'s between the
calls so that there is actually a difference in the time stamps. For
atime and mtime I can probably also use utime() to first reset the times
and avoid the sleep, but for ctime there seems to be no way to avoid it.


Best,

   -Nikolaus

--
 »Time flies like an arrow, fruit flies like a Banana.«

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises
looking to deploy the next generation of Solaris that includes the latest
innovations from Sun and the OpenSource community. Download a copy and
enjoy capabilities such as Networking, Storage and Virtualization.
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel