Quantcast

Large write broken into 4096-byte chunks

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Large write broken into 4096-byte chunks

David Shaw
If I write a program like:

  #include <stdio.h>

  int main(int argc,char *argv[])
  {
    FILE *file;
    char buf[32768];

    file=fopen("data","w");
    fwrite(buf,32768,1,file);
    fclose(file);
  }

and run it against a fuse filesystem, instead of the single 32k write,
the filesystem program sees 8 4k writes.  Is there any reason for
this?

A number of times in the past the idea of a large write has come up
where fuse would buffer writes until some large size was reached and
then do one large write all at once.  Understood that that is not
possible, but this is something else - there should be no buffering
needed here, since the original write was large.

David


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large write broken into 4096-byte chunks

Edwin Olson

>    fwrite(buf,32768,1,file);
>
>  
>
God knows what kind of buffering stdio is doing...

But aside from that, isn't it the case that the linux VFS layer
internally deals with page-sized chunks of data, which on x86 means 4kB?
Perhaps the gurus can confirm/deny!

-Ed



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large write broken into 4096-byte chunks

David Shaw
On Fri, Jun 17, 2005 at 06:00:50PM -0400, Edwin Olson wrote:
>
> >   fwrite(buf,32768,1,file);
> >
> >
> >
> God knows what kind of buffering stdio is doing...

In this case, none, since strace shows the fwrite maps to a single
write() call of size 32768.

> But aside from that, isn't it the case that the linux VFS layer
> internally deals with page-sized chunks of data, which on x86 means 4kB?
> Perhaps the gurus can confirm/deny!

Possible, but this doesn't happen when reading via fuse.  On when
writing.

David


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Large write broken into 4096-byte chunks

Miklos Szeredi
> > But aside from that, isn't it the case that the linux VFS layer
> > internally deals with page-sized chunks of data, which on x86 means 4kB?
> > Perhaps the gurus can confirm/deny!
>
> Possible, but this doesn't happen when reading via fuse.  On when
> writing.

It actually does happen when reading, only the single page chunks are
again aggregated by the readahead logic into a larger read.  But that
will not necessarily be the exact same chunk that the user requested
with read().

With writing, the problem with aggregation, is that there's no way to
know when a page being written is the last one belonging to a single
write() syscall.  If that information was available, it would be
possible to implement your suggestion.

Miklos



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Loading...