Should I announce a new file system?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Should I announce a new file system?

Johannes Zarl
Hi,

I've written a FUSE filesystem that allows different users to have different
views on the same directory [1].

Now I have two questions:

Should I announce the filesystem anywhere and/or add it to the list [2] in the
wiki?

Is there any well-documented way of testing a filesystem, or at least some
documentation on what test can/should fail for a specific file system type?

I've seen the corresponding FAQ entry [3], and I've tried the pjd-fstest
testsuite, but that gives some failed tests even for my regular XFS file
system, and the error messages when I run it on my self-written filesystem are
not really helpful. For now I've resorted to some basic testing by hand, but
some basic confidence test would "feel better" ;-)

Cheers,
  Johannes


[1] https://github.com/jzarl/unsharedfs
[2] http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FileSystems
[3]
http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FAQ#How_do_I_test_my_new_file_system.3F

------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Jean-Pierre André
Hi

Johannes Zarl wrote:

> Hi,
>
> I've written a FUSE filesystem that allows different users to have different
> views on the same directory [1].
>[...]
> I've seen the corresponding FAQ entry [3], and I've tried the pjd-fstest
> testsuite, but that gives some failed tests even for my regular XFS file
> system, and the error messages when I run it on my self-written filesystem are
> not really helpful. For now I've resorted to some basic testing by hand, but
> some basic confidence test would "feel better" ;-)

On http://sourceforge.net/p/ntfs-3g/pjd-fstest/ci/master/tree/
I have updated the pjd-fstest test suite to take into account
other file systems, among them xfs. xfs appears to have a
behavior different from ext234 in a few situations which Posix
documents as open to variations, or when it does not say what
the behavior should be.

You still have to dig into the scripts to find out what the
failed tests mean...

Jean-Pierre


------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Johannes Zarl
Hi,

On Wednesday, 21. May 2014, 16:14:45, Jean-Pierre André wrote:
> On http://sourceforge.net/p/ntfs-3g/pjd-fstest/ci/master/tree/
> I have updated the pjd-fstest test suite to take into account
> other file systems, among them xfs. xfs appears to have a
> behavior different from ext234 in a few situations which Posix
> documents as open to variations, or when it does not say what
> the behavior should be.

Thanks. The new version of the testsuite seems to work on xfs (the xacl tests
fail, but I guess that's ok when one doesn't use acls).

> You still have to dig into the scripts to find out what the
> failed tests mean...

Ok. I had still hoped that there was some kind of documentation to get me
started...

How do I read e.g. the following test?

>> expect 0 -u 65534 -g 65534 create ${n0}/${n1} 0464

Do I correctly assume that the first param (0) is the expected return value,
and the -u and -g specify the user and group the test is run under?

If -u and -g change the uid/gid under which the test is executed, it would
explain why so many tests don't work with my unsharedfs...

Thanks,
  Johannes

------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Jean-Pierre André
Hi again,

Johannes Zarl wrote:
> Hi,

> Thanks. The new version of the testsuite seems to work on xfs (the xacl tests
> fail, but I guess that's ok when one doesn't use acls).

If the command "if setfacl -m 'm::5' somefile" returns
an error, the xacl tests are skipped. With xfs the acls may
be enabled depending on the mount option "acl" and/or
formatting options.

>> You still have to dig into the scripts to find out what the
>> failed tests mean...
>
> Ok. I had still hoped that there was some kind of documentation to get me
> started...

Beware of the comment lines such as "# 18" which are
supposed to indicate the test number. These numbers obviously
do not take into account the tests skipped under condition,
so they may be wrong.

>
> How do I read e.g. the following test?
>
>>> expect 0 -u 65534 -g 65534 create ${n0}/${n1} 0464
>
> Do I correctly assume that the first param (0) is the expected return value,
> and the -u and -g specify the user and group the test is run under?

Exactly.

Jean-Pierre


------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Bernd Schubert-2
In reply to this post by Johannes Zarl
On 05/21/2014 04:43 PM, Johannes Zarl wrote:

> Hi,
>
> On Wednesday, 21. May 2014, 16:14:45, Jean-Pierre André wrote:
>> On http://sourceforge.net/p/ntfs-3g/pjd-fstest/ci/master/tree/
>> I have updated the pjd-fstest test suite to take into account
>> other file systems, among them xfs. xfs appears to have a
>> behavior different from ext234 in a few situations which Posix
>> documents as open to variations, or when it does not say what
>> the behavior should be.
>
> Thanks. The new version of the testsuite seems to work on xfs (the xacl tests
> fail, but I guess that's ok when one doesn't use acls).
>
>> You still have to dig into the scripts to find out what the
>> failed tests mean...
>
> Ok. I had still hoped that there was some kind of documentation to get me
> started...
>
> How do I read e.g. the following test?
>
>>> expect 0 -u 65534 -g 65534 create ${n0}/${n1} 0464
>
> Do I correctly assume that the first param (0) is the expected return value,
> and the -u and -g specify the user and group the test is run under?

Yes.

>
> If -u and -g change the uid/gid under which the test is executed, it would
> explain why so many tests don't work with my unsharedfs...

Again yes.


As pjd-fstests is not suitable for parallel file systems, one of our
students converted it to a server/client model in python (several
clients connect to the server, which sends commands to a random client).
I don't think that already works perfectly as Bagavathi always tested
with single server/client. But even in its current state it should help
you to figure out which command actually fails (which is IMHO not
easywith pjd).
I hope I will have eventually some time to improve the project later
this week or early next week...

https://bitbucket.org/aakef/posix_fstests


Cheers,
Bernd




------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Shachar Sharon-2
In reply to this post by Johannes Zarl
Hi Johannes,
I am not aware of any well-documented open-source framework for testing
filesystems. However, I discovered that both git and postgresql have
excellent test-suits, which may help you to test your file-system.
Try running those two commands on your file-system:

$ git clone https://github.com/git/git && cd git && make -j 8 && make test

$ git clone git://git.postgresql.org/git/postgresql.git && cd postgresql &&
./configure && make check


Don't be surprised if you discover unexpected bugs ;)
- Shachar.



On Wed, May 21, 2014 at 3:34 PM, Johannes Zarl <[hidden email]> wrote:

> Hi,
>
> I've written a FUSE filesystem that allows different users to have
> different
> views on the same directory [1].
>
> Now I have two questions:
>
> Should I announce the filesystem anywhere and/or add it to the list [2] in
> the
> wiki?
>
> Is there any well-documented way of testing a filesystem, or at least some
> documentation on what test can/should fail for a specific file system type?
>
> I've seen the corresponding FAQ entry [3], and I've tried the pjd-fstest
> testsuite, but that gives some failed tests even for my regular XFS file
> system, and the error messages when I run it on my self-written filesystem
> are
> not really helpful. For now I've resorted to some basic testing by hand,
> but
> some basic confidence test would "feel better" ;-)
>
> Cheers,
>   Johannes
>
>
> [1] https://github.com/jzarl/unsharedfs
> [2] http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FileSystems
> [3]
>
> http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FAQ#How_do_I_test_my_new_file_system.3F
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing platform
> available
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> fuse-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/fuse-devel
>
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Johannes Zarl
Hi Shachar,

Am 22.05.2014, 08:09:31 schrieb Shachar Sharon:
> I am not aware of any well-documented open-source framework for testing
> filesystems. However, I discovered that both git and postgresql have
> excellent test-suits, which may help you to test your file-system.
> Try running those two commands on your file-system:
>
> $ git clone https://github.com/git/git && cd git && make -j 8 && make test
>
> $ git clone git://git.postgresql.org/git/postgresql.git && cd postgresql &&
> ./configure && make check

Thanks! This seems to be well-suited for my needs.


> Don't be surprised if you discover unexpected bugs ;)

Already found some bugs ;-)

Cheers,
  Johannes

------------------------------------------------------------------------------
The best possible search technologies are now affordable for all companies.
Download your FREE open source Enterprise Search Engine today!
Our experts will assist you in its installation for $59/mo, no commitment.
Test it for FREE on our Cloud platform anytime!
http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Johannes Zarl
In reply to this post by Bernd Schubert-2
Hi,

Am 21.05.2014, 17:16:10 schrieb Bernd Schubert:

> As pjd-fstests is not suitable for parallel file systems, one of our
> students converted it to a server/client model in python (several
> clients connect to the server, which sends commands to a random client).
> I don't think that already works perfectly as Bagavathi always tested
> with single server/client. But even in its current state it should help
> you to figure out which command actually fails (which is IMHO not
> easywith pjd).
> I hope I will have eventually some time to improve the project later
> this week or early next week...
>
> https://bitbucket.org/aakef/posix_fstests

Thanks for the link. Expecially with the builtin descriptions this seems like
a good alternative to pjd.

For now I will stick to Shachar's suggestion of running the git/postfix test
suites, though. Since the (normally reasonable) assumption that all users have
the same view on the file system does not hold for unsharedfs, I guess a
"standard" file system test suite won't work very well for me...

Cheers,
  Johannes


------------------------------------------------------------------------------
The best possible search technologies are now affordable for all companies.
Download your FREE open source Enterprise Search Engine today!
Our experts will assist you in its installation for $59/mo, no commitment.
Test it for FREE on our Cloud platform anytime!
http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Should I announce a new file system?

Fabrice Bauzac
FYI I have just added this testsuite as well as Shachar's into the FUSE FAQ.

2014-05-26 13:37 GMT+02:00 Johannes Zarl <[hidden email]>:

> Hi,
>
> Am 21.05.2014, 17:16:10 schrieb Bernd Schubert:
>> As pjd-fstests is not suitable for parallel file systems, one of our
>> students converted it to a server/client model in python (several
>> clients connect to the server, which sends commands to a random client).
>> I don't think that already works perfectly as Bagavathi always tested
>> with single server/client. But even in its current state it should help
>> you to figure out which command actually fails (which is IMHO not
>> easywith pjd).
>> I hope I will have eventually some time to improve the project later
>> this week or early next week...
>>
>> https://bitbucket.org/aakef/posix_fstests
>
> Thanks for the link. Expecially with the builtin descriptions this seems like
> a good alternative to pjd.
>
> For now I will stick to Shachar's suggestion of running the git/postfix test
> suites, though. Since the (normally reasonable) assumption that all users have
> the same view on the file system does not hold for unsharedfs, I guess a
> "standard" file system test suite won't work very well for me...
>
> Cheers,
>   Johannes
>
>
> ------------------------------------------------------------------------------
> The best possible search technologies are now affordable for all companies.
> Download your FREE open source Enterprise Search Engine today!
> Our experts will assist you in its installation for $59/mo, no commitment.
> Test it for FREE on our Cloud platform anytime!
> http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
> _______________________________________________
> fuse-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/fuse-devel

------------------------------------------------------------------------------
The best possible search technologies are now affordable for all companies.
Download your FREE open source Enterprise Search Engine today!
Our experts will assist you in its installation for $59/mo, no commitment.
Test it for FREE on our Cloud platform anytime!
http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel