Possible FUSE kernel issue

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Possible FUSE kernel issue

Antonio SJ Musumeci
I'm getting reports from a few of my users that as of Linux 4.9.0 they are having random crashes.

They are using mergerfs 2.19.0 with libfuse 2.9.7. When they downgraded to Linux 4.8.14 or lower the crashing stopped.

From a strace you can see a short read from /dev/fuse and then libfuse calls _exit.

Any ideas?

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Miklos Szeredi
On Mon, Feb 13, 2017 at 2:42 AM, Antonio SJ Musumeci
<[hidden email]> wrote:
> I'm getting reports from a few of my users that as of Linux 4.9.0 they are
> having random crashes.
>
> They are using mergerfs 2.19.0 with libfuse 2.9.7. When they downgraded to
> Linux 4.8.14 or lower the crashing stopped.
>
> From a strace you can see a short read from /dev/fuse and then libfuse calls
> _exit.

Is this reproducible?

Can I please have a look at the strace?

Thanks,
Miklos

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Antonio SJ Musumeci

On Mon, Feb 13, 2017 at 4:00 AM, Miklos Szeredi <[hidden email]> wrote:
On Mon, Feb 13, 2017 at 2:42 AM, Antonio SJ Musumeci
<[hidden email]> wrote:
> I'm getting reports from a few of my users that as of Linux 4.9.0 they are
> having random crashes.
>
> They are using mergerfs 2.19.0 with libfuse 2.9.7. When they downgraded to
> Linux 4.8.14 or lower the crashing stopped.
>
> From a strace you can see a short read from /dev/fuse and then libfuse calls
> _exit.

Is this reproducible?

Can I please have a look at the strace?

Thanks,
Miklos


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Antonio SJ Musumeci
I got another report from someone using Linux 4.9.7. It "will suddenly drop the mount without any entries in journalctl, kernel or otherwise."

On Mon, Feb 13, 2017 at 8:13 AM, Antonio SJ Musumeci <[hidden email]> wrote:

On Mon, Feb 13, 2017 at 4:00 AM, Miklos Szeredi <[hidden email]> wrote:
On Mon, Feb 13, 2017 at 2:42 AM, Antonio SJ Musumeci
<[hidden email]> wrote:
> I'm getting reports from a few of my users that as of Linux 4.9.0 they are
> having random crashes.
>
> They are using mergerfs 2.19.0 with libfuse 2.9.7. When they downgraded to
> Linux 4.8.14 or lower the crashing stopped.
>
> From a strace you can see a short read from /dev/fuse and then libfuse calls
> _exit.

Is this reproducible?

Can I please have a look at the strace?

Thanks,
Miklos



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Miklos Szeredi
On Wed, Feb 15, 2017 at 12:11 AM, Antonio SJ Musumeci
<[hidden email]> wrote:
> I got another report from someone using Linux 4.9.7. It "will suddenly drop
> the mount without any entries in journalctl, kernel or otherwise."
>
> On Mon, Feb 13, 2017 at 8:13 AM, Antonio SJ Musumeci <[hidden email]>
> wrote:
>>
>> Yes, its reproducible. Apparently at will.
>>
>> https://github.com/trapexit/mergerfs/files/768926/mergerfs.strace.gz

Thanks for the trace.

Can you (someone) please try this patch on 4.9 and report back if it
makes the crash go away?

Thanks,
Miklos

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel

fuse-fix-uninitialized-flags-in-pipe_buffer.patch (582 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Antonio SJ Musumeci
I'll forward the patch along and ask if they can try it out. An additional bit of info. One of the users has said it only seems to crash while reading. Writes don't seem to trigger the exits.

On Wed, Feb 15, 2017 at 5:54 AM, Miklos Szeredi <[hidden email]> wrote:
On Wed, Feb 15, 2017 at 12:11 AM, Antonio SJ Musumeci
<[hidden email]> wrote:
> I got another report from someone using Linux 4.9.7. It "will suddenly drop
> the mount without any entries in journalctl, kernel or otherwise."
>
> On Mon, Feb 13, 2017 at 8:13 AM, Antonio SJ Musumeci <[hidden email]>
> wrote:
>>
>> Yes, its reproducible. Apparently at will.
>>
>> https://github.com/trapexit/mergerfs/files/768926/mergerfs.strace.gz

Thanks for the trace.

Can you (someone) please try this patch on 4.9 and report back if it
makes the crash go away?

Thanks,
Miklos


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Antonio SJ Musumeci
Err... Nope. Still happening. 

On Feb 15, 2017 9:38 PM, "Antonio SJ Musumeci" <[hidden email]> wrote:
So far so good. They are going to continue testing and report back in another day or so.

On Wed, Feb 15, 2017 at 7:58 AM, Antonio SJ Musumeci <[hidden email]> wrote:
I'll forward the patch along and ask if they can try it out. An additional bit of info. One of the users has said it only seems to crash while reading. Writes don't seem to trigger the exits.

On Wed, Feb 15, 2017 at 5:54 AM, Miklos Szeredi <[hidden email]> wrote:
On Wed, Feb 15, 2017 at 12:11 AM, Antonio SJ Musumeci
<[hidden email]> wrote:
> I got another report from someone using Linux 4.9.7. It "will suddenly drop
> the mount without any entries in journalctl, kernel or otherwise."
>
> On Mon, Feb 13, 2017 at 8:13 AM, Antonio SJ Musumeci <[hidden email]>
> wrote:
>>
>> Yes, its reproducible. Apparently at will.
>>
>> https://github.com/trapexit/mergerfs/files/768926/mergerfs.strace.gz

Thanks for the trace.

Can you (someone) please try this patch on 4.9 and report back if it
makes the crash go away?

Thanks,
Miklos



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Miklos Szeredi
On Thu, Feb 16, 2017 at 1:19 PM, Antonio SJ Musumeci
<[hidden email]> wrote:
> Err... Nope. Still happening.

So, I have no idea.  Here's debug patch that might shed some light on this...

Please apply this patch and send dmesg output after the crash.

Thanks,
Miklos

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel

debug.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Antonio SJ Musumeci

On Thu, Feb 16, 2017 at 9:03 AM, Miklos Szeredi <[hidden email]> wrote:
On Thu, Feb 16, 2017 at 1:19 PM, Antonio SJ Musumeci
<[hidden email]> wrote:
> Err... Nope. Still happening.

So, I have no idea.  Here's debug patch that might shed some light on this...

Please apply this patch and send dmesg output after the crash.

Thanks,
Miklos


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Antonio SJ Musumeci
Was any of that info helpful?

On Sun, Feb 26, 2017 at 7:26 PM, Antonio SJ Musumeci <[hidden email]> wrote:

On Thu, Feb 16, 2017 at 9:03 AM, Miklos Szeredi <[hidden email]> wrote:
On Thu, Feb 16, 2017 at 1:19 PM, Antonio SJ Musumeci
<[hidden email]> wrote:
> Err... Nope. Still happening.

So, I have no idea.  Here's debug patch that might shed some light on this...

Please apply this patch and send dmesg output after the crash.

Thanks,
Miklos



------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible FUSE kernel issue

Miklos Szeredi
On Tue, Mar 7, 2017 at 1:01 PM, Antonio SJ Musumeci <[hidden email]> wrote:
> Was any of that info helpful?

No, there's no output from the debug patch.

Need all lines beggining with "f_d_s_r" and "p_r" to be able to
determine what happened.

Thanks,
Miklos

>
> On Sun, Feb 26, 2017 at 7:26 PM, Antonio SJ Musumeci <[hidden email]>
> wrote:
>>
>> http://pastebin.com/mEQnZgpB
>>
>> https://github.com/trapexit/mergerfs/issues/370#issuecomment-282598664
>>
>> On Thu, Feb 16, 2017 at 9:03 AM, Miklos Szeredi <[hidden email]> wrote:
>>>
>>> On Thu, Feb 16, 2017 at 1:19 PM, Antonio SJ Musumeci
>>> <[hidden email]> wrote:
>>> > Err... Nope. Still happening.
>>>
>>> So, I have no idea.  Here's debug patch that might shed some light on
>>> this...
>>>
>>> Please apply this patch and send dmesg output after the crash.
>>>
>>> Thanks,
>>> Miklos
>>
>>
>

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
--
fuse-devel mailing list
To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel
Loading...