Mastering ADF/HDF Images with xdftool

My Amiga cross development environment on my Mac is getting really useful now: with vamos running the SAS C compiler I can create Amiga binaries with ease. With the binaries in place I want to try them on the real machine, too. For my trusty old Amiga 500, I still use disks to transfer the data. So I create an ADF image with my files on it and either use my kryoflux setup to write a real disk or write a virtual HFE disk image on an SD card to be used with the HXC2001 floppy emulator.

While building the code is done automatically in a Makefile, the disk image creation still involves manual steps including launching an UAE emulator to create the disk image. Hmm, I thought, an ADF file mastering tool (like mkisofs is used for CDs) would be a great tool!! Adding this to my Makefile would fully automates my build cycle… I was aware of ADFlib as being the portable library for ADF manipulation and I had a look there, but I did not found a mastering tool. Some more googling didn’t show me a similar tool – so again – I was left on my own and had to create the tool myself 😉

As a result xdftool was born and added to my amitools tool set! I started writing a small tool using the Python binding of ADFlib to create an ADF image and copy files there. This worked really quickly but soon I found the limitations of this library: the mastering part is only partially supported and essential things like setting all meta infos of a file were missing (or I didn’t find them ;). In the end I dropped the ADFlib approach and started to build a FS library for Amiga OFS/FFS from scratch… Thanks to the excellent ADF Disk Format FAQ I soon had all necessary information available and some hours later the first code files written.

The lib is done now and included in amitools as “fs” module. In its first incarnation it already has an impressive feature set:

  • full object-oriented API in Python
  • supports ADF and HDF containers
  • supports all OFS/FFS modes including international and dircache
  • read/write files/dir trees from/to host file system
  • multi-layer support: work on block level or on FS level
  • query and modify all parameters found in the file system including comments, protection flags and all time stamps in tick resolution
  • supports unpacking/packing/repacking of full images into host file system with meta db files to store information not available on host file system

xdftool now supports all features of the library and while it started as a mastering tool for ADFs it is now a full featured command line tool to work with all kinds of ADF and HDF images… The remainder of this post gives you a short tutorial what you can do with xdftool:

First of all install xdftool. It is a part of amitools so have a look at my amitools page for installation details. It is handy to pack the top-level dir of amitools into your PATH so you have all tools available in your shell at any place.

In the following I will use a Workbench 3.1 disk image called wb31.adf which was taken from Cloanto’s Amiga Forever Pack. You can use any other disk image as well. Simply adjust the paths if necessary. Host files are taken from the top level directory of amitools.

This tutorial only shows a sub set of functions available in xdftool. Have a look at the xdftool Homepage for a full reference of commands.

Look at a Disk

> xdftool wb31.adf list
Workbench3.1                                    VOLUME  --------  06.07.1986 14:32:19 t08 
 C                                                 DIR  ----rwed  22.06.1991 02:13:25 t18 
 AddBuffers                                        444  --p-rwed  06.07.1986 14:39:06 t31 
 AddDataTypes                                     5880  --p-rwed  06.07.1986 14:39:06 t37 
 Assign                                           3220  --p-rwed  06.07.1986 14:39:06 t43 
 Avail                                             736  --p-rwed  06.07.1986 14:39:06 t49 
 BindDrivers                                      1420  ----rwed  06.07.1986 14:39:07 t04 
 Break                                             432  --p-rwed  06.07.1986 14:39:07 t16 
 ChangeTaskPri                                     460  --p-rwed  06.07.1986 14:39:07 t21 
 ConClip                                          2432  --p-rwed  06.07.1986 14:39:07 t29 
...
T                                                  DIR  ----rwed  22.06.1991 02:06:08 t25 
 Utilities                                         DIR  ----rwed  06.07.1986 14:39:05 t07 
 Clock                                           13856  ----rwed  06.07.1986 14:39:04 t31 
 Clock.info                                        590  ----rw-d  06.07.1986 14:39:04 t37 
 More                                            12752  --p-rwed  06.07.1986 14:39:04 t43 
 MultiView                                       28836  --p-rwed  06.07.1986 14:39:05 t01 
 MultiView.info                                    861  ----rw-d  06.07.1986 14:39:05 t11 
 Utilities.info                                    632  ----rw-d  06.07.1986 14:39:06 t14 
 WBStartup                                         DIR  ----rwed  06.07.1986 14:39:05 t19 
 WBStartup.info                                    632  ----rw-d  06.07.1986 14:39:06 t21 
Blocks:    data:     1496, fs:      199, sum:     1695, ratio=88.26, disk total:     1760
Bytes:     data:   765952, fs:   101888, sum:   867840, ratio=88.26, disk total:   901120
File:      data:   726742, ratio=94.88

Ok, first command shows you whats on a disk! Note the calling convention in xdftool: always pass the image file name first then the command with parameters. You see the complete directory tree of the volume inside the disk image. Each entry shows you the size, protection flags, modification time stamps (with ticks) and if available the file comment. At the end a summary on block usage is given.

Create a disk and write to it

> xdftool mydisk.adf format MyDisk + boot install + list
MyDisk                                         VOLUME  --------  14.01.2012 21:49:13 t00 
Blocks:    data:        0, fs:        1, sum:        1, ratio=0.00, disk total:     1760
Bytes:     data:        0, fs:      512, sum:      512, ratio=0.00, disk total:   901120
File:      data:        0, ratio=0.00

With this command creating an empty formatted image is a no-brainer… Note that you can combine multiple commands in a single call to xdftool by separating them with a plus sign. Creating a hard disk image is also very easy:

> xdftool myhd.hdf format Work 10M

Here a size (or a disk geometry) needs to be specified.

You can put some files from your host on it with the ‘write’ command:

> xdftool mydisk.adf write README + list
test                                            VOLUME  --------  14.01.2012 21:51:42 t00 
 README                                           3728  ----rwed  14.01.2012 21:51:42 t00 
Blocks:    data:        8, fs:        2, sum:       10, ratio=80.00, disk total:     1760
Bytes:     data:     4096, fs:     1024, sum:     5120, ratio=80.00, disk total:   901120
File:      data:     3728, ratio=91.02

This will write the file README from the host’s current directory to the volume directory of the image. You can also write a full directory tree to the image:

> xdftool mydisk.adf write doc + list
test                                            VOLUME  --------  14.01.2012 21:53:20 t00 
 doc                                               DIR  ----rwed  14.01.2012 21:53:20 t00 
   xdftool.txt                                   18798  ----rwed  14.01.2012 21:53:20 t00 
Blocks:    data:       39, fs:        3, sum:       42, ratio=92.86, disk total:     1760
Bytes:     data:    19968, fs:     1536, sum:    21504, ratio=92.86, disk total:   901120
File:      data:    18798, ratio=94.14

Now you can alter the protect flags, the modification time or set a comment:

> xdftool mydisk.adf protect doc rwe + time doc "12.01.1991 12:00:00 t11" + comment doc "my doc comment" + list
test                                            VOLUME  --------  14.01.2012 21:53:20 t00 
 doc                                               DIR  ----rwe-  12.01.1991 12:00:00 t11  my doc comment
   xdftool.txt                                   18798  ----rwed  14.01.2012 21:53:20 t00 
Blocks:    data:       39, fs:        3, sum:       42, ratio=92.86, disk total:     1760
Bytes:     data:    19968, fs:     1536, sum:    21504, ratio=92.86, disk total:   901120
File:      data:    18798, ratio=94.14

With these commands at hand you can now easily write a Makefile entry that creates an ADF image and places all your files on it.

But xdftool can more:

Unpacking and Packing Images

The unpack command extracts all files and directories from a disk image and creates a corresponding directory structure on your host’s file system. Additionally a MetaDB file will be created with all flags of the Amiga file system that cannot be represented in the host file system, namely protect flags, timestamps with ticks and comments:

> xdftool wb31.adf unpack .
> ls Workbench3.1*
Workbench3.1.blkdev    Workbench3.1.bootcode  Workbench3.1.xdfmeta

Workbench3.1:
C/              Devs.info       Expansion.info  Prefs/          S/              T/              WBStartup/
Classes/        Disk.info       L/              Prefs.info      System/         Utilities/      WBStartup.info
Devs/           Expansion/      Libs/           Rexxc/          System.info     Utilities.info

You see a directory named after the volume inside the image was created with all files and dirs of the image. Additionally the MetaDB file *.xdfmeta, the boot code in *.bootcode and the description of the disk’s block device geometry *.blkdev was written.

With these files in place you can always recreate a new disk image that is equivalent to the original image (from the FS perspective):

> xdftool newimg.adf pack Workbench3.1 + list
Workbench3.1                                    VOLUME  --------  06.07.1986 14:32:19 t08  
 C                                                 DIR  ----rwed  22.06.1991 02:13:25 t18  
 AddBuffers                                        444  --p-rwed  06.07.1986 14:39:06 t31  
 AddDataTypes                                     5880  --p-rwed  06.07.1986 14:39:06 t37  
 Assign                                           3220  --p-rwed  06.07.1986 14:39:06 t43  
...
   MultiView.info                                  861  ----rw-d  06.07.1986 14:39:05 t11  
 Utilities.info                                    632  ----rw-d  06.07.1986 14:39:06 t14  
 WBStartup                                         DIR  ----rwed  06.07.1986 14:39:05 t19  
 WBStartup.info                                    632  ----rw-d  06.07.1986 14:39:06 t21  
Blocks:    data:     1496, fs:      199, sum:     1695, ratio=88.26, disk total:     1760
Bytes:     data:   765952, fs:   101888, sum:   867840, ratio=88.26, disk total:   901120
File:      data:   726742, ratio=94.88

Note the correct time stamps and protect flags after packing…

With this approach you can quickly unpack an image, modify some files on your host file system and pack the image again.

Also mastering full images is now easy: create a Volume directory from scratch, place all your files in it and if you need special flags then write a *.xdfmeta file. Call xdftool pack to create your image. Done!

Repacking Images

Another nice operation in xdftool is repack: This essentially combines a unpack command on an existing image with a pack command on a new image. So you can rebuild the contents of the whole volume of the old image and write a fresh new FS structure. This will recreate the FS and re-aligns all file data blocks to be in one place. Originally this command helps me to test the integrity of amitools fs library but it may be useful for you, too:

> xdftool wb31.adf repack new.adf

In combination with a hard disk image repack also allows you to create a larger target disk, so this command essentially allows you to “grow” your Amiga hard disk by repacking its image to a larger one:

> xdftool old.hdf repack new.hdf 10M

You can even repack a disk image into a hard disk image:

> xdftool wb31.adf repack wb31.hdf 10M

That’s it… If you want to play more with xdftool have a look at the xdftool page!

Have Fun! And if you find bugs in the rather fresh code then please don’t hesitate to report them… Thanks!

 

 

Leave a Reply