Posts tagged with "make" - Jay Caines-Gooby 2010-08-06T00:14:12+01:00 Jay Caines-Gooby Scripting hybrid CDROM & DVD burning with hdiutil and rake 2010-08-06T00:14:12+01:00 2010-08-06T01:48:36+01:00 <p><a href="">Max</a> was writing some rake tasks today and it reminded me to finish off this post which has sat unfinished for months.</p> <p>Bob&#8217;s been porting <a href="">Charanga&#8217;s music-teaching desktop software</a> from PC to Macs. The port is based on the work we&#8217;ve done in the past couple of years for our <a href=''>online products</a> and means that we can now have an online offering as well as PC and Mac desktop products, all built from the same codebase.</p> <p>We&#8217;ve <a href=";record_id=10180">initially released</a> <a href=";record_id=10181">three</a> <a href=";record_id=10182">products</a> with a further eight to follow.</p> <p>Each of these 11 products will come as either a hybrid <span class="caps">DVD</span> or <span class="caps">CDROM</span>, with both the PC and Mac version on, but only visible to relevant platform. Lots of CD burning products for Macs out there make it easy to burn these kind of ROMs, but the big problem is that they all need to be made manually. And with 11 different products, that&#8217;s 11 different manual processes, any one mistake of which could ruin the master that we&#8217;re sending off to the publisher.</p> <p>It occurred to me that we deploy <a href="">our</a> <a href="">web</a> <a href="">apps</a> with a single invocation:</p> <pre name="code" class="shell:nogutter:nocontrols"> cap deploy STAGE=production </pre> <p>So why not do the same with the burning of the CDROMs? Entirely automate the process so there&#8217;s no room for manual error&#8230;</p> <h3>Rake</h3> <p>Rake &#8211; <a href="">Ruby Make</a> &#8211; operates on a rakefile which defines lists of tasks, with optional requisite tasks that must first be completed. Given that building and burning the ROMs consists of a bunch of identical steps, differentiated only by the files that need to go on the relevant product&#8217;s <span class="caps">CDROM</span> or <span class="caps">DVD</span>, it sounds like an ideal tool, so let&#8217;s go ahead and build a skeleton rakefile&#8230;</p> <p>There are a bunch of files that are common to all the products, plus product specific files. These get pulled out of subversion (yes, yes, we&#8217;re only just migrating to git), copied into the product filestructure, the PC content gets added, the hybrid <span class="caps">ISO</span> image gets created and then we use this to physically burn the <span class="caps">ROM</span>.</p> <p>If we make each preceding step a prerequisite of the parent task, we can break the the steps down into nice self-contained pieces and have a single task invoke all the others below it.</p> <p>Ultimately, I wanted to be able to stick a <span class="caps">DVD</span> or <span class="caps">CDROM</span> into the drive and then call:</p> <pre name="code" class="shell:nogutter:nocontrols"> rake burn_electric_guitar_coach_dvd </pre> <p>And have a finished hybrid <span class="caps">DVD</span> pop out, fresh off the press.</p> <h3>Break it down</h3> <p>The tasks in the rakefile are roughly as follows:</p> <pre name="code" class="ruby"> # Define various constants # The source repository REPOSITORY_URL="" # Where we'll do all this stuff BUILD_DIR = File.expand_path "/Users/jay/Work/music coach" # The copy of the remote repository we'll use locally CACHED_COPY = "#{BUILD_DIR}/svn-cached-copy" # shared by all products COMMON_CONTENT = "'#{PRODUCTION_FOLDER}/help'" PRODUCTS = { # specific properties for each product :guitar_deluxe =&gt; { # Mac content :volume_name =&gt; "Play Electric Guitar", :app_folder_name =&gt; "Play Electric Guitar v3.0", :logo =&gt; "guitar deluxe.jpg", :modules =&gt; "First Lessons For Guitar", "Guitar Improver", "Guitar Songs And Styles", "Solo Guitar Performance Pieces", "Master Rock Power Chords", "Chord miner"] # PC content :pc_iso =&gt; "guitar_deluxe.cdr", :pc_iso_volume_name =&gt; "GuitarDeluxe", # details of which files to hide from a PC on a Mac and vice versa :hide_hfs =&gt; "{Common,Player,program files,Redist,System32,*.exe,*.inf,*.msi,*.ini}", :hide_joliet =&gt; "{.background,.DS_Store,.Trashes,,.fseventsd,Play Piano v3.0,Applications}" }, :electric_guitar_deluxe =&gt; { # ... }, :piano_deluxe =&gt; { # ... }, :play_piano =&gt; { # ... } # and so on for 7 other products } # A couple of helpers... # Input helper - gets input from user def ask message puts message STDIN.gets.chomp end # Symol helper - converts a string to a symbol # "Blah blah foo".symbolize = :blah_blah_foo class String def symbolize self.downcase.split(" ").join("_").to_sym end end # Now the tasks themselves # The default task (runs when rake is called without arguments) task :default =&gt; :create_repository # The create_repository task - builds a local copy of the repository for us to work from desc "Create a cached copy folder where the repository will reside which we can then svn export the installer files from" task :create_repository do # the production files unless File.exists?("#{CACHED_COPY}") puts "Creating initial cached copy of the repository" svn_user ||= ask("Enter your svn username: ") svn_password ||= ask("Enter your svn password: ") sh "svn checkout --username #{svn_user} --password #{svn_password} '#{REPOSITORY_URL}' '#{CACHED_COPY}'" end end desc "Update the cached copy of the respository to get latest versions of files" task :update_repository =&gt; [:create_repository] do puts "Updating #{topics_product} production files" sh "cd '#{CACHED_COPY}'; #{SVN_PATH}/svn update" end # Be DRY about the task creation and use some string to symbol magic to dynamically create the tasks # # This makes three tasks per product (11 products = 33 tasks :) # # # 1. build_#{topics_product}_dmg (with a prerequisite on 1.) # # 2. build_#{topics_product}_dvd (with a prerequisite on 2.) # # 3. burn_#{topics_product}_dvd (with a prerequisite on 3.) PRODUCTS.each do |topics_product, data| desc "Build #{topics_product} for Mac .dmg" task "build_#{topics_product}_dmg".symbolize =&gt; [:update_repository] do # We need to clean up .dmg and any old build folders # and make sure no other dmg of the same name is mounted sh "sudo umount -f '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}") sh "sudo rm -rf '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}") sh "rm '/tmp/#{PRODUCTS[topics_product][:dmg]}'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:dmg]}") sh "rm '/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg") # Take the read-only master .dmg that has the backgrounds, .DS_Store and folder stubs # and make a copy of it to /tmp, then resize the copy so we can add our content, then mount it sh "hdiutil convert '#{CACHED_COPY}/development/mac installer/#{PRODUCTS[topics_product][:dmg]}' -format UDRW -o '/tmp/#{PRODUCTS[topics_product][:dmg]}'" sh "hdiutil resize -size 4g '/tmp/#{PRODUCTS[topics_product][:dmg]}'; hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5" # The new, writable dmg is now mounted at '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}' # and it's where we'll assemble the rest of the dmg tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}/#{PRODUCTS[topics_product][:app_folder_name]}" # Sort the permissions out (99 is the magic OS X user and group that appears to be owned by the current user when viewed; i.e. my uid is 6, but when I ls -l a file owned 99:99 it appears as 6:6) sh "sudo chown -R 99:99 '#{tmp_product_dir}'" sh "sudo chmod -R 777 '#{tmp_product_dir}'" # Export the product modules to make this specific .dmg PRODUCTS[topics_product][:modules].each do |product_module| sh "cd '#{CACHED_COPY}'; #{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/#{product_module}' '#{tmp_product_dir}/modules/';" end # export the help sh "cd '#{CACHED_COPY}';#{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/help' '#{tmp_product_dir}/Production File System/help'" end # Right we're done making the .dmg so unmount it. The file will remain in /tmp sh "hdiutil detach -force '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" end desc "Create the hybrid PC &amp; Mac .iso file for #{topics_product}" task "build_#{topics_product}_dvd".symbolize =&gt; "build_#{topics_product}".symbolize do # Create a Hybrid DVD image containing the contents of the PC .iso and the contents of the Mac .dmg # # get rid of any old tmp files sh "sudo rm -rf '/tmp/#{PRODUCTS[topics_product][:pc_iso_volume_name]}'" # mount the PC .iso sh "hdiutil attach '#{PRODUCTS[topics_product][:pc_iso]}'; sleep 5" # mount the Mac .dmg - this is where we'll copy the sh "hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5" tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}" # Copy the contents of the PC .iso to the mounted Mac .dmg (which is why we resized it to 4Gb earlier) # hdiutil needs to operate on a mounted volume to successfully create a hybrid iso sh "ditto '/Volumes/#{PRODUCTS[topics_product][:pc_iso_volume_name]}' '#{tmp_product_dir}'" # - unmount the PC .iso sh "sudo umount -f /Volumes/'#{PRODUCTS[topics_product][:pc_iso_volume_name]}'" # Remove any previous hyrid .iso prior to making this one sh "rm -f '/tmp/hybrid.iso'" # Make the hybrid iso # exlude PC files from the Mac, and exclude Mac files from the PC sh "hdiutil makehybrid -o /tmp/hybrid.iso '#{tmp_product_dir}' \ -hfs -iso -joliet \ -hide-hfs '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_hfs]}' \ -hide-joliet '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}' \ -hide-iso '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}'" end desc "Burn the .iso file for #{topics_product} to DVD" task "burn_#{topics_product}_dvd".symbolize =&gt; "build_#{topics_product}_dvd".symbolize do # Burn the DVD # That really long device is the external burner's firmware address (we're using an external writer) # I can never remember how to get this. Use # # hdiutil burn -list | grep IOService # # to list your device sh "hdiutil burn -device IOService:/AppleACPIPlatformExpert/PCI0@0/AppleACPIPCI/PCIB@1E/IOPCI2PCIBridge/FRWR@3/AppleFWOHCI/IOFireWireController/IOFireWireDevice@d04b990c04c4d1/IOFireWireUnit/IOFireWireSBP2Target/IOFireWireSBP2LUN/com_apple_driver_Oxford_Semi_FW934_DSA/IOSCSIPeripheralDeviceNub/IOSCSIPeripheralDeviceType05/IODVDServices /tmp/hybrid.iso" end end </pre> <p>The end result of this is that we end up with 11 <code>burn_&lt;a product name&gt;_dvd</code> tasks, which each invoke in turn</p> <ol> <li><code>update_repository</code></li> <li><code>build_&lt;a product name&gt;_dmg</code></li> <li><code>build_&lt;a product name&gt;_dvd</code></li> </ol> <p>and ends up with a burnt, hybrid <span class="caps">DVD</span> being made for you.</p> <p>The critical part happens at line 163 where the Mac .dmg is mounted under /Volumes, and is writeable, has the PC content written to it. It seems that hdiutil only likes mounted images when creating hybrid images. I experimented with various other options (using directories in /tmp, etc) but for the hybrid image to built correctly, it seems this is your only choice. The -hide switches list via globs, which files to hide from each filesystem.</p>