treating partitioned files as disks

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

treating partitioned files as disks

Eric S. Johansson
I have a project (ipcop) where I am trying to create a flash image and
then modify it slightly multiple times for each instance where it is used.

the code currently though some really ugly things in that it creates a
file for every partition so I can format, copy data etc. and then uses
dd to merge everything together into one big happy flash image.

Needless to say, as soon as you modify anything, "things break".

I am looking for some way to treat a file as if it were a block device
so I could format, impose filesystem, and then mount it so I can copy
all of the files in place and hit it with grub to make it bootable.

Is this something fuse can do easily?  Already?  I think this technique
would also be useful in producing partitioned CD-ROM images.

Appreciate your assistance.

---eric



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: treating partitioned files as disks

Joshua J. Berry
On Sunday 31 July 2005 15:59, Eric S. Johansson wrote:
> I am looking for some way to treat a file as if it were a block device
> so I could format, impose filesystem, and then mount it so I can copy
> all of the files in place and hit it with grub to make it bootable.

mount -o loop

(or, if you want a plain block device, losetup)

> Is this something fuse can do easily?  Already?  I think this technique
> would also be useful in producing partitioned CD-ROM images.

FUSE doesn't do anything with raw block devices.  You probably want the
loopback device (/dev/loopX).

-- Josh

--
Joshua J. Berry

"I haven't lost my mind -- it's backed up on tape somewhere."
    -- /usr/games/fortune

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: treating partitioned files as disks

Ian C. Blenke
In reply to this post by Eric S. Johansson
Eric S. Johansson wrote:

> I have a project (ipcop) where I am trying to create a flash image and
> then modify it slightly multiple times for each instance where it is
> used.
>
> the code currently though some really ugly things in that it creates a
> file for every partition so I can format, copy data etc. and then uses
> dd to merge everything together into one big happy flash image.
>
> Needless to say, as soon as you modify anything, "things break".
>
> I am looking for some way to treat a file as if it were a block device
> so I could format, impose filesystem, and then mount it so I can copy
> all of the files in place and hit it with grub to make it bootable.
>
> Is this something fuse can do easily?  Already?  I think this
> technique would also be useful in producing partitioned CD-ROM images.

Take a look at FiST's UnionFS (http://filesystems.org/project-unionfs.html)

FUSE is a great way to implement a quick and dirty filesystem to do
whatever you're looking to do.

There are, however, some projects that probably already address your
problem.

 - Ian C. Blenke <[hidden email]>



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: treating partitioned files as disks

Eric S. Johansson
In reply to this post by Joshua J. Berry
Joshua J. Berry wrote:
> On Sunday 31 July 2005 15:59, Eric S. Johansson wrote:
>
>>I am looking for some way to treat a file as if it were a block device
>>so I could format, impose filesystem, and then mount it so I can copy
>>all of the files in place and hit it with grub to make it bootable.
>
>
> mount -o loop

only works if the file has a single partition and as a filesystem.

>
> (or, if you want a plain block device, losetup)

kind of sort of works.  Can only partition consistently with fdisk and
not sfdisk.  the latter is required if you are attempting to write a
script to partition a file as a disc.  The biggest problem obviously is
that a file does not have sectors, heads, cylinders.

>>Is this something fuse can do easily?  Already?  I think this technique
>>would also be useful in producing partitioned CD-ROM images.
>
>
> FUSE doesn't do anything with raw block devices.  You probably want the
> loopback device (/dev/loopX).

it's beginning to sound like I really need a loopback device that
actually handles different partitions in the same way that hda can
handle multiple partitions (hda1, hda2, ...) and can impose h/s/c
semantics on a file.

Thanks.  Appreciate the help.

---eric



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: treating partitioned files as disks

Eric S. Johansson
In reply to this post by Ian C. Blenke
Ian C. Blenke wrote:
> Take a look at FiST's UnionFS (http://filesystems.org/project-unionfs.html)

seems like union fs is a great way to produced merged images of existing
filesystem which is not exactly what I'm trying to do although it may
sound like it.  Poor description on my part.
>
> FUSE is a great way to implement a quick and dirty filesystem to do
> whatever you're looking to do.

as I said earlier, I am coming to understand that fuse is the wrong
level from where I need to be.  I really need something like fuse at a
block level.


> There are, however, some projects that probably already address your
> problem.

cruel man cruel.  ;-) giving me a hint without a pointer.  Other than
union fs which is not what I need, suggestions?

since this is getting way off topic for the list, feel free to mail
directly.

---eric



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: Re: treating partitioned files as disks

Miklos Szeredi
> as I said earlier, I am coming to understand that fuse is the wrong
> level from where I need to be.  I really need something like fuse at a
> block level.

There's NBD (network block device), which doesn't have a nice API like
FUSE, but the protocol is probably quite simple.

Miklos


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: treating partitioned files as disks

Ian C. Blenke
In reply to this post by Ian C. Blenke
Eric S. Johansson wrote:

> Ian C. Blenke wrote:
>
>> Eric S. Johansson wrote:
>>
>>> I have a project (ipcop) where I am trying to create a flash image
>>> and then modify it slightly multiple times for each instance where
>>> it is used.
>>>
>>> the code currently though some really ugly things in that it creates
>>> a file for every partition so I can format, copy data etc. and then
>>> uses dd to merge everything together into one big happy flash image.
>>>
>>> Needless to say, as soon as you modify anything, "things break".
>>>
>>> I am looking for some way to treat a file as if it were a block
>>> device so I could format, impose filesystem, and then mount it so I
>>> can copy all of the files in place and hit it with grub to make it
>>> bootable.
>>>
>>> Is this something fuse can do easily?  Already?  I think this
>>> technique would also be useful in producing partitioned CD-ROM images.
>>
>>
>> Take a look at FiST's UnionFS
>> (http://filesystems.org/project-unionfs.html)
>
>
> seems like union fs is a great way to produced merged images of
> existing filesystem which is not exactly what I'm trying to do
> although it may sound like it.  Poor description on my part.

Ah. Many CD based distros tend to use something like UnionFS to have an
"overlay" RAM or Flash device that stores whatever changes you make to
the underlying read-only filesystem. If I were making a bootable CD for
my linux distro, a UnionFS approach would let me store filesytem changes
to something like JFFS while making it look like the entire ISO9660/UDF
filesystem is writable.

>> FUSE is a great way to implement a quick and dirty filesystem to do
>> whatever you're looking to do.
>
> as I said earlier, I am coming to understand that fuse is the wrong
> level from where I need to be.  I really need something like fuse at a
> block level.

FUSE is a filesystem abstraction. If you're primarily dealing with a
filesytem with POSIX semantics, FUSE is a great way of quickly
implementing a userspace filesystem.

>> There are, however, some projects that probably already address your
>> problem.
>
> cruel man cruel.  ;-) giving me a hint without a pointer.  Other than
> union fs which is not what I need, suggestions?
>
> since this is getting way off topic for the list, feel free to mail
> directly.

Joshua's loopback tips sounds like what you are looking for, only with
partition support.

It sounds like you're looking for a "partitioned loopback device".
Plugging this into google, I find:

    http://lwn.net/Articles/110426/

That's a patch for a 2.6 kernel that supports loopback mounting a
partitioned block device or file.

Barring that, you can always calculate the offset of the partition,
following this HOWTO:

    http://edseek.com/~jasonb/articles/linux_loopback.html

There is a link to a "enhanced loopback device" from NASA that solves
your problem.

These were the first couple of hits from  Google, and sound like exactly
what you are looking for.

- Ian C. Blenke <[hidden email]> <[hidden email]> http://ian.blenke.com
 



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel
Reply | Threaded
Open this post in threaded view
|

Re: treating partitioned files as disks

Eric S. Johansson
Ian C. Blenke wrote:
> Joshua's loopback tips sounds like what you are looking for, only with
> partition support.

they were indeed useful.
>
> It sounds like you're looking for a "partitioned loopback device".
> Plugging this into google, I find:
>
>    http://lwn.net/Articles/110426/

sent e-mail to the author asking if it made into the kernel or is just
another patch I'll have to maintain to infinity and beyond


> Barring that, you can always calculate the offset of the partition,
> following this HOWTO:
>
>    http://edseek.com/~jasonb/articles/linux_loopback.html

definitely useful.

>
> There is a link to a "enhanced loopback device" from NASA that solves
> your problem.
>
> These were the first couple of hits from  Google, and sound like exactly
> what you are looking for.

knew about the ones you've mentioned here.  Again, a most of those seem
to be suffering from bit rot.  I think my most likely candidate is NBD.
  Seems rather twisted but what pact, I'd rather put my energy into
something interesting than beating my head against the wall.

thank you all so very very much for your helpful suggestions.  they were
truly appreciated.

---eric

>
> - Ian C. Blenke <[hidden email]>
> <[hidden email]> http://ian.blenke.com
>
>
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
fuse-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/fuse-devel