Jay Caines-Gooby

Back in 5 mins

Max was writing some rake tasks today and it reminded me to finish off this post which has sat unfinished for months.

Bob’s been porting Charanga’s music-teaching desktop software from PC to Macs. The port is based on the work we’ve done in the past couple of years for our online products and means that we can now have an online offering as well as PC and Mac desktop products, all built from the same codebase.

We’ve initially released three products with a further eight to follow.

Each of these 11 products will come as either a hybrid DVD or CDROM, with both the PC and Mac version on, but only visible to relevant platform. Lots of CD burning products for Macs out there make it easy to burn these kind of ROMs, but the big problem is that they all need to be made manually. And with 11 different products, that’s 11 different manual processes, any one mistake of which could ruin the master that we’re sending off to the publisher.

It occurred to me that we deploy our web apps with a single invocation:

	cap deploy STAGE=production

So why not do the same with the burning of the CDROMs? Entirely automate the process so there’s no room for manual error…


Rake – Ruby Make – operates on a rakefile which defines lists of tasks, with optional requisite tasks that must first be completed. Given that building and burning the ROMs consists of a bunch of identical steps, differentiated only by the files that need to go on the relevant product’s CDROM or DVD, it sounds like an ideal tool, so let’s go ahead and build a skeleton rakefile…

There are a bunch of files that are common to all the products, plus product specific files. These get pulled out of subversion (yes, yes, we’re only just migrating to git), copied into the product filestructure, the PC content gets added, the hybrid ISO image gets created and then we use this to physically burn the ROM.

If we make each preceding step a prerequisite of the parent task, we can break the the steps down into nice self-contained pieces and have a single task invoke all the others below it.

Ultimately, I wanted to be able to stick a DVD or CDROM into the drive and then call:

	rake burn_electric_guitar_coach_dvd

And have a finished hybrid DVD pop out, fresh off the press.

Break it down

The tasks in the rakefile are roughly as follows:

# Define various constants

# The source repository

# Where we'll do all this stuff
BUILD_DIR = File.expand_path "/Users/jay/Work/music coach" 

# The copy of the remote repository we'll use locally
CACHED_COPY = "#{BUILD_DIR}/svn-cached-copy"

# shared by all products


  # specific properties for each product
  :guitar_deluxe => { 

    # Mac content
    :volume_name => "Play Electric Guitar",
    :app_folder_name => "Play Electric Guitar v3.0",
    :logo => "guitar deluxe.jpg",    
    :modules => "First Lessons For Guitar", "Guitar Improver", "Guitar Songs And Styles", "Solo Guitar Performance Pieces", "Master Rock Power Chords", "Chord miner"]
    # PC content
    :pc_iso => "guitar_deluxe.cdr",
    :pc_iso_volume_name => "GuitarDeluxe",
    # details of which files to hide from a PC on a Mac and vice versa
    :hide_hfs => "{Common,Player,program files,Redist,System32,*.exe,*.inf,*.msi,*.ini}",
    :hide_joliet => "{.background,.DS_Store,.Trashes,.com.apple.timemachine.supported,.fseventsd,Play Piano v3.0,Applications}"
  :electric_guitar_deluxe => {
    # ...
  :piano_deluxe => {
    # ...
  :play_piano => {
    # ...
  # and so on for 7 other products

# A couple of helpers...
# Input helper - gets input from user
def ask message
puts message

# Symol helper - converts a string to a symbol
# "Blah blah foo".symbolize = :blah_blah_foo
class String
  def symbolize
    self.downcase.split(" ").join("_").to_sym

# Now the tasks themselves

# The default task (runs when rake is called without arguments)
task :default => :create_repository

# The create_repository task - builds a local copy of the repository for us to work from 
desc "Create a cached copy folder where the repository will reside which we can then svn export the installer files from"
task :create_repository do 
  # the production files
  unless File.exists?("#{CACHED_COPY}")
    puts "Creating initial cached copy of the repository"
    svn_user ||= ask("Enter your svn username: ")
    svn_password ||= ask("Enter your svn password: ")
    sh "svn checkout --username #{svn_user} --password #{svn_password} '#{REPOSITORY_URL}' '#{CACHED_COPY}'"

desc "Update the cached copy of the respository to get latest versions of files"
task  :update_repository => [:create_repository] do

  puts "Updating #{topics_product} production files"
  sh "cd '#{CACHED_COPY}'; #{SVN_PATH}/svn update"
# Be DRY about the task creation and use some string to symbol magic to dynamically create the tasks
# This makes three tasks per product (11 products = 33 tasks :)
# 1. build_#{topics_product}_dmg (with a prerequisite on 1.)
# 2. build_#{topics_product}_dvd (with a prerequisite on 2.)
# 3. burn_#{topics_product}_dvd (with a prerequisite on 3.)

PRODUCTS.each do |topics_product, data|
  desc "Build #{topics_product} for Mac .dmg"
  task "build_#{topics_product}_dmg".symbolize => [:update_repository] do

    # We need to clean up .dmg and any old build folders
    # and make sure no other dmg of the same name is mounted
    sh "sudo umount -f '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}")
    sh "sudo rm -rf '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}")
    sh "rm '/tmp/#{PRODUCTS[topics_product][:dmg]}'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:dmg]}")
    sh "rm '/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg")    
    # Take the read-only master .dmg that has the backgrounds, .DS_Store and folder stubs
    # and make a copy of it to /tmp, then resize the copy so we can add our content, then mount it
    sh "hdiutil convert '#{CACHED_COPY}/development/mac installer/#{PRODUCTS[topics_product][:dmg]}' -format UDRW -o '/tmp/#{PRODUCTS[topics_product][:dmg]}'"
    sh "hdiutil resize -size 4g '/tmp/#{PRODUCTS[topics_product][:dmg]}'; hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5"
    # The new, writable dmg is now mounted at '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'
    # and it's where we'll assemble the rest of the dmg
    tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}/#{PRODUCTS[topics_product][:app_folder_name]}"
    # Sort the permissions out (99 is the magic OS X user and group that appears to be owned by the current user when viewed; i.e. my uid is 6, but when I ls -l a file owned 99:99 it appears as 6:6)
    sh "sudo chown -R 99:99 '#{tmp_product_dir}'"
    sh "sudo chmod -R 777 '#{tmp_product_dir}'"
    # Export the product modules to make this specific .dmg
    PRODUCTS[topics_product][:modules].each do |product_module|
      sh "cd '#{CACHED_COPY}';  #{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/#{product_module}' '#{tmp_product_dir}/modules/';"
    # export the help
    sh "cd '#{CACHED_COPY}';#{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/help' '#{tmp_product_dir}/Production File System/help'"   
    # Right we're done making the .dmg so unmount it. The file will remain in /tmp
    sh "hdiutil detach -force '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'"    
  desc "Create the hybrid PC & Mac .iso file for #{topics_product}"
  task "build_#{topics_product}_dvd".symbolize => "build_#{topics_product}".symbolize do
    # Create a Hybrid DVD image containing the contents of the PC .iso and the contents of the Mac .dmg
    # get rid of any old tmp files
    sh "sudo rm -rf '/tmp/#{PRODUCTS[topics_product][:pc_iso_volume_name]}'"

    # mount the PC .iso
    sh "hdiutil attach '#{PRODUCTS[topics_product][:pc_iso]}'; sleep 5"

    # mount the Mac .dmg - this is where we'll copy the 
    sh "hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5"

    tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}"
    # Copy the contents of the PC .iso to the mounted Mac .dmg (which is why we resized it to 4Gb earlier)
    # hdiutil needs to operate on a mounted volume to successfully create a hybrid iso
    sh "ditto '/Volumes/#{PRODUCTS[topics_product][:pc_iso_volume_name]}' '#{tmp_product_dir}'"    

    # - unmount the PC .iso
    sh "sudo umount -f /Volumes/'#{PRODUCTS[topics_product][:pc_iso_volume_name]}'"

    # Remove any previous hyrid .iso prior to making this one
    sh "rm -f '/tmp/hybrid.iso'"
    # Make the hybrid iso
    # exlude PC files from the Mac, and exclude Mac files from the PC
    sh "hdiutil makehybrid -o /tmp/hybrid.iso '#{tmp_product_dir}' \
                           -hfs -iso -joliet \
                           -hide-hfs    '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_hfs]}' \
                           -hide-joliet '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}' \
                           -hide-iso    '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}'"
  desc "Burn the .iso file for #{topics_product} to DVD"
  task "burn_#{topics_product}_dvd".symbolize => "build_#{topics_product}_dvd".symbolize do
    # Burn the DVD                          
    # That really long device is the external burner's firmware address (we're using an external writer)
    # I can never remember how to get this. Use 
    #   hdiutil burn -list | grep IOService
    # to list your device
    sh "hdiutil burn -device IOService:/AppleACPIPlatformExpert/PCI0@0/AppleACPIPCI/PCIB@1E/IOPCI2PCIBridge/FRWR@3/AppleFWOHCI/IOFireWireController/IOFireWireDevice@d04b990c04c4d1/IOFireWireUnit/IOFireWireSBP2Target/IOFireWireSBP2LUN/com_apple_driver_Oxford_Semi_FW934_DSA/IOSCSIPeripheralDeviceNub/IOSCSIPeripheralDeviceType05/IODVDServices /tmp/hybrid.iso"    


The end result of this is that we end up with 11 burn_<a product name>_dvd tasks, which each invoke in turn

  1. update_repository
  2. build_<a product name>_dmg
  3. build_<a product name>_dvd

and ends up with a burnt, hybrid DVD being made for you.

The critical part happens at line 163 where the Mac .dmg is mounted under /Volumes, and is writeable, has the PC content written to it. It seems that hdiutil only likes mounted images when creating hybrid images. I experimented with various other options (using directories in /tmp, etc) but for the hybrid image to built correctly, it seems this is your only choice. The -hide switches list via globs, which files to hide from each filesystem.


New comment

required, won't be displayed


Don't type anything here unless you're an evil robot:

And especially don't type anything here:

Basic XHTML (including links) is allowed, just don't try anything fishy. Your comment will be auto-formatted unless you use your own <p> tags for formatting. You're also welcome to use Textile.

Copyright © 2011 Jay Caines-Gooby. All rights reserved.
Powered by Thoth.