Big writes bigger than 64 kb libfuse 2.9.7

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Big writes bigger than 64 kb libfuse 2.9.7

José Lourenço
Hi there,

I have been trying to use FUSE’s big_write mode with requests larger than 64kb with no success.

I am using libfuse version 2.9.7 and just found this on your mailing list https://sourceforge.net/p/fuse/mailman/message/35418253/
It seems that writes are limited to 64kb in version 2.9.7.

Can you confirm on this?
Is there any way I can increase this value without modifying the kernel? I am unable to do so in my environment.

Best regards,
José Lourenço

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Big writes bigger than 64 kb libfuse 2.9.7

Yue Li
hi Jose,

There are two approaches for doing this, the first one is to modify the
kernel and fuse library's parameters (only three parameters need to be
changed, very simple). This worked for me very well, and my fuse now
write in 1MB chunks.

The other way is to locally buffer data yourself in the write() handler,
and have a monitoring thread watching the buffer.

When a thread calls write, it fills buffer and checks whether the buffer
reaches the write size you want (e.g. 1MB), if limit is reached, the
thread flushes the buffer by doing pwrite. Otherwise, it simply exists
but return the size of the data (of course it's faking the write to the
user). The monitoring thread periodically wakes up to check write
time-out: If there's a time-out, it flushes whatever data into buffer.
(This is for flushing the last piece of the data into the disk).

Note that you need to use something like a hash table to manage buffers
for different files as they might be written at the same time. A buffer
shall only be used for one file exclusively. This also worked extremely
well for me.

Best,

Yue


On 3/20/17 5:02 AM, José Lourenço wrote:

> Hi there,
>
> I have been trying to use FUSE’s big_write mode with requests larger than 64kb with no success.
>
> I am using libfuse version 2.9.7 and just found this on your mailing list https://sourceforge.net/p/fuse/mailman/message/35418253/ <https://sourceforge.net/p/fuse/mailman/message/35418253/>
> It seems that writes are limited to 64kb in version 2.9.7.
>
> Can you confirm on this?
> Is there any way I can increase this value without modifying the kernel? I am unable to do so in my environment.
>
> Best regards,
> José Lourenço
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel