Problem with FUSE and IBM's GPFS

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with FUSE and IBM's GPFS

Mathew Binkley
One of our users has tried to use FUSE/EncFS on our cluster to encrypt
export-controlled information and is having problems (ie doesn't work at
all).  I have replicated the problem successfully, and it seems to be
due to a GPFS problem, or at least a GPFS-FUSE interaction.

To explain our setup, we have a large Beowulf cluster running Whitebox
3, kernel 2.4.21-32.0.1.  Our cluster uses IBM's GPFS to export /home to
the cluster.  We just recently (within the past month) installed the
GPFS system; previously we used NFS to export /home.

I have tested three test cases:

* "New" kernel (32.0.1) v. "Old" (15.0.4), both with GPFS:  both fail
* "New" kernel", both with GPFS but one without AFS:  both fail
* "New" kernel with GPFS v "New" without:  with fails, without passes

These cases, combined with the fact that it worked fine before migrating
to GPFS, suggest that either GPFS has a bug, or FUSE doesn't like GPFS.

The problem is experienced when trying to mount an encrypted partition.
  I have copied a sample below.  You get a "permission denied" when
trying to cd or ls the mounted directory.  The problem happens
irrespective of whether "crypt" or "plain" exist in either a GPFS
filesystem or on a local ext3 filesystem.

---

[binklemj@vmpsched binklemj]$ cd /tmp/binklemj/

[binklemj@vmpsched binklemj]$ ls
crypt  plain

[binklemj@vmpsched binklemj]$ encfs /tmp/binklemj/crypt /tmp/binklemj/plain
EncFS Password:

[binklemj@vmpsched binklemj]$ ls
crypt  plain

[binklemj@vmpsched binklemj]$ cd plain
-bash: cd: plain: Permission denied

[binklemj@vmpsched binklemj]$ ls -alh
ls: plain: Permission denied
total 12K
drwxr-xr-x    4 binklemj accre        4.0K Sep 28 14:28 .
drwxrwxrwt    9 root     root         4.0K Sep 28 15:22 ..
drwx------    2 binklemj accre        4.0K Sep 28 14:28 crypt

---

Would anyone be willing to take a look at this for me?  If so, email me
and I'll set up an account for you.


Mat


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Problem with FUSE and IBM's GPFS

Miklos Szeredi
> The problem is experienced when trying to mount an encrypted partition.
>   I have copied a sample below.  You get a "permission denied" when
> trying to cd or ls the mounted directory.  The problem happens
> irrespective of whether "crypt" or "plain" exist in either a GPFS
> filesystem or on a local ext3 filesystem.

You might try to look at:

 - strace of 'ls'
 - log output of encfs (-d option)
 - strace of 'encfs'

to see where the error originates from.

Miklos


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Problem with FUSE and IBM's GPFS

Mathew Binkley
Miklos Szeredi wrote:

>>The problem is experienced when trying to mount an encrypted partition.
>>  I have copied a sample below.  You get a "permission denied" when
>>trying to cd or ls the mounted directory.  The problem happens
>>irrespective of whether "crypt" or "plain" exist in either a GPFS
>>filesystem or on a local ext3 filesystem.
>
> You might try to look at:
>
>  - strace of 'ls'
>  - log output of encfs (-d option)
>  - strace of 'encfs'

I've placed the outputs from those steps at:

    http://www.accre.vanderbilt.edu/docs/gpfs-fuse-debug/

The strace from ls looks like it has some weird stuff going on, but I
don't know enough to be certain.  Any help is appreciated.


Mat


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Problem with FUSE and IBM's GPFS

Valient Gough
In reply to this post by Mathew Binkley
On Wednesday 28 September 2005 13:57, Mathew Binkley wrote:
> I have tested three test cases:
>
> * "New" kernel (32.0.1) v. "Old" (15.0.4), both with GPFS:  both fail
> * "New" kernel", both with GPFS but one without AFS:  both fail
> * "New" kernel with GPFS v "New" without:  with fails, without passes

This is interesting.  When you don't build GPFS into the kernel, then it
works?

> Would anyone be willing to take a look at this for me?  If so, email me
> and I'll set up an account for you.

I could take a look (I'm the encfs author), but it sounds like there won't be
anything I can do at the encfs level.  Sounds like it may be a kernel-level
issue.

Valient

attachment0 (196 bytes) Download Attachment