Posts tagged with "charanga" - Jay Caines-Gooby 2010-08-06T00:14:12+01:00 Jay Caines-Gooby Scripting hybrid CDROM & DVD burning with hdiutil and rake 2010-08-06T00:14:12+01:00 2010-08-06T01:48:36+01:00 <p><a href="">Max</a> was writing some rake tasks today and it reminded me to finish off this post which has sat unfinished for months.</p> <p>Bob&#8217;s been porting <a href="">Charanga&#8217;s music-teaching desktop software</a> from PC to Macs. The port is based on the work we&#8217;ve done in the past couple of years for our <a href=''>online products</a> and means that we can now have an online offering as well as PC and Mac desktop products, all built from the same codebase.</p> <p>We&#8217;ve <a href=";record_id=10180">initially released</a> <a href=";record_id=10181">three</a> <a href=";record_id=10182">products</a> with a further eight to follow.</p> <p>Each of these 11 products will come as either a hybrid <span class="caps">DVD</span> or <span class="caps">CDROM</span>, with both the PC and Mac version on, but only visible to relevant platform. Lots of CD burning products for Macs out there make it easy to burn these kind of ROMs, but the big problem is that they all need to be made manually. And with 11 different products, that&#8217;s 11 different manual processes, any one mistake of which could ruin the master that we&#8217;re sending off to the publisher.</p> <p>It occurred to me that we deploy <a href="">our</a> <a href="">web</a> <a href="">apps</a> with a single invocation:</p> <pre name="code" class="shell:nogutter:nocontrols"> cap deploy STAGE=production </pre> <p>So why not do the same with the burning of the CDROMs? Entirely automate the process so there&#8217;s no room for manual error&#8230;</p> <h3>Rake</h3> <p>Rake &#8211; <a href="">Ruby Make</a> &#8211; operates on a rakefile which defines lists of tasks, with optional requisite tasks that must first be completed. Given that building and burning the ROMs consists of a bunch of identical steps, differentiated only by the files that need to go on the relevant product&#8217;s <span class="caps">CDROM</span> or <span class="caps">DVD</span>, it sounds like an ideal tool, so let&#8217;s go ahead and build a skeleton rakefile&#8230;</p> <p>There are a bunch of files that are common to all the products, plus product specific files. These get pulled out of subversion (yes, yes, we&#8217;re only just migrating to git), copied into the product filestructure, the PC content gets added, the hybrid <span class="caps">ISO</span> image gets created and then we use this to physically burn the <span class="caps">ROM</span>.</p> <p>If we make each preceding step a prerequisite of the parent task, we can break the the steps down into nice self-contained pieces and have a single task invoke all the others below it.</p> <p>Ultimately, I wanted to be able to stick a <span class="caps">DVD</span> or <span class="caps">CDROM</span> into the drive and then call:</p> <pre name="code" class="shell:nogutter:nocontrols"> rake burn_electric_guitar_coach_dvd </pre> <p>And have a finished hybrid <span class="caps">DVD</span> pop out, fresh off the press.</p> <h3>Break it down</h3> <p>The tasks in the rakefile are roughly as follows:</p> <pre name="code" class="ruby"> # Define various constants # The source repository REPOSITORY_URL="" # Where we'll do all this stuff BUILD_DIR = File.expand_path "/Users/jay/Work/music coach" # The copy of the remote repository we'll use locally CACHED_COPY = "#{BUILD_DIR}/svn-cached-copy" # shared by all products COMMON_CONTENT = "'#{PRODUCTION_FOLDER}/help'" PRODUCTS = { # specific properties for each product :guitar_deluxe =&gt; { # Mac content :volume_name =&gt; "Play Electric Guitar", :app_folder_name =&gt; "Play Electric Guitar v3.0", :logo =&gt; "guitar deluxe.jpg", :modules =&gt; "First Lessons For Guitar", "Guitar Improver", "Guitar Songs And Styles", "Solo Guitar Performance Pieces", "Master Rock Power Chords", "Chord miner"] # PC content :pc_iso =&gt; "guitar_deluxe.cdr", :pc_iso_volume_name =&gt; "GuitarDeluxe", # details of which files to hide from a PC on a Mac and vice versa :hide_hfs =&gt; "{Common,Player,program files,Redist,System32,*.exe,*.inf,*.msi,*.ini}", :hide_joliet =&gt; "{.background,.DS_Store,.Trashes,,.fseventsd,Play Piano v3.0,Applications}" }, :electric_guitar_deluxe =&gt; { # ... }, :piano_deluxe =&gt; { # ... }, :play_piano =&gt; { # ... } # and so on for 7 other products } # A couple of helpers... # Input helper - gets input from user def ask message puts message STDIN.gets.chomp end # Symol helper - converts a string to a symbol # "Blah blah foo".symbolize = :blah_blah_foo class String def symbolize self.downcase.split(" ").join("_").to_sym end end # Now the tasks themselves # The default task (runs when rake is called without arguments) task :default =&gt; :create_repository # The create_repository task - builds a local copy of the repository for us to work from desc "Create a cached copy folder where the repository will reside which we can then svn export the installer files from" task :create_repository do # the production files unless File.exists?("#{CACHED_COPY}") puts "Creating initial cached copy of the repository" svn_user ||= ask("Enter your svn username: ") svn_password ||= ask("Enter your svn password: ") sh "svn checkout --username #{svn_user} --password #{svn_password} '#{REPOSITORY_URL}' '#{CACHED_COPY}'" end end desc "Update the cached copy of the respository to get latest versions of files" task :update_repository =&gt; [:create_repository] do puts "Updating #{topics_product} production files" sh "cd '#{CACHED_COPY}'; #{SVN_PATH}/svn update" end # Be DRY about the task creation and use some string to symbol magic to dynamically create the tasks # # This makes three tasks per product (11 products = 33 tasks :) # # # 1. build_#{topics_product}_dmg (with a prerequisite on 1.) # # 2. build_#{topics_product}_dvd (with a prerequisite on 2.) # # 3. burn_#{topics_product}_dvd (with a prerequisite on 3.) PRODUCTS.each do |topics_product, data| desc "Build #{topics_product} for Mac .dmg" task "build_#{topics_product}_dmg".symbolize =&gt; [:update_repository] do # We need to clean up .dmg and any old build folders # and make sure no other dmg of the same name is mounted sh "sudo umount -f '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}") sh "sudo rm -rf '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}") sh "rm '/tmp/#{PRODUCTS[topics_product][:dmg]}'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:dmg]}") sh "rm '/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg") # Take the read-only master .dmg that has the backgrounds, .DS_Store and folder stubs # and make a copy of it to /tmp, then resize the copy so we can add our content, then mount it sh "hdiutil convert '#{CACHED_COPY}/development/mac installer/#{PRODUCTS[topics_product][:dmg]}' -format UDRW -o '/tmp/#{PRODUCTS[topics_product][:dmg]}'" sh "hdiutil resize -size 4g '/tmp/#{PRODUCTS[topics_product][:dmg]}'; hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5" # The new, writable dmg is now mounted at '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}' # and it's where we'll assemble the rest of the dmg tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}/#{PRODUCTS[topics_product][:app_folder_name]}" # Sort the permissions out (99 is the magic OS X user and group that appears to be owned by the current user when viewed; i.e. my uid is 6, but when I ls -l a file owned 99:99 it appears as 6:6) sh "sudo chown -R 99:99 '#{tmp_product_dir}'" sh "sudo chmod -R 777 '#{tmp_product_dir}'" # Export the product modules to make this specific .dmg PRODUCTS[topics_product][:modules].each do |product_module| sh "cd '#{CACHED_COPY}'; #{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/#{product_module}' '#{tmp_product_dir}/modules/';" end # export the help sh "cd '#{CACHED_COPY}';#{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/help' '#{tmp_product_dir}/Production File System/help'" end # Right we're done making the .dmg so unmount it. The file will remain in /tmp sh "hdiutil detach -force '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" end desc "Create the hybrid PC &amp; Mac .iso file for #{topics_product}" task "build_#{topics_product}_dvd".symbolize =&gt; "build_#{topics_product}".symbolize do # Create a Hybrid DVD image containing the contents of the PC .iso and the contents of the Mac .dmg # # get rid of any old tmp files sh "sudo rm -rf '/tmp/#{PRODUCTS[topics_product][:pc_iso_volume_name]}'" # mount the PC .iso sh "hdiutil attach '#{PRODUCTS[topics_product][:pc_iso]}'; sleep 5" # mount the Mac .dmg - this is where we'll copy the sh "hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5" tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}" # Copy the contents of the PC .iso to the mounted Mac .dmg (which is why we resized it to 4Gb earlier) # hdiutil needs to operate on a mounted volume to successfully create a hybrid iso sh "ditto '/Volumes/#{PRODUCTS[topics_product][:pc_iso_volume_name]}' '#{tmp_product_dir}'" # - unmount the PC .iso sh "sudo umount -f /Volumes/'#{PRODUCTS[topics_product][:pc_iso_volume_name]}'" # Remove any previous hyrid .iso prior to making this one sh "rm -f '/tmp/hybrid.iso'" # Make the hybrid iso # exlude PC files from the Mac, and exclude Mac files from the PC sh "hdiutil makehybrid -o /tmp/hybrid.iso '#{tmp_product_dir}' \ -hfs -iso -joliet \ -hide-hfs '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_hfs]}' \ -hide-joliet '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}' \ -hide-iso '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}'" end desc "Burn the .iso file for #{topics_product} to DVD" task "burn_#{topics_product}_dvd".symbolize =&gt; "build_#{topics_product}_dvd".symbolize do # Burn the DVD # That really long device is the external burner's firmware address (we're using an external writer) # I can never remember how to get this. Use # # hdiutil burn -list | grep IOService # # to list your device sh "hdiutil burn -device IOService:/AppleACPIPlatformExpert/PCI0@0/AppleACPIPCI/PCIB@1E/IOPCI2PCIBridge/FRWR@3/AppleFWOHCI/IOFireWireController/IOFireWireDevice@d04b990c04c4d1/IOFireWireUnit/IOFireWireSBP2Target/IOFireWireSBP2LUN/com_apple_driver_Oxford_Semi_FW934_DSA/IOSCSIPeripheralDeviceNub/IOSCSIPeripheralDeviceType05/IODVDServices /tmp/hybrid.iso" end end </pre> <p>The end result of this is that we end up with 11 <code>burn_&lt;a product name&gt;_dvd</code> tasks, which each invoke in turn</p> <ol> <li><code>update_repository</code></li> <li><code>build_&lt;a product name&gt;_dmg</code></li> <li><code>build_&lt;a product name&gt;_dvd</code></li> </ol> <p>and ends up with a burnt, hybrid <span class="caps">DVD</span> being made for you.</p> <p>The critical part happens at line 163 where the Mac .dmg is mounted under /Volumes, and is writeable, has the PC content written to it. It seems that hdiutil only likes mounted images when creating hybrid images. I experimented with various other options (using directories in /tmp, etc) but for the hybrid image to built correctly, it seems this is your only choice. The -hide switches list via globs, which files to hide from each filesystem.</p> Rendering bitmaps from PDFs at non-native sizes 2009-11-17T16:32:28+00:00 2010-12-08T10:26:47+00:00 <p>Charanga&#8217;s primary instrumental teaching resources use a synchronised score and animated instrument to indicate which notes are played during the piece.</p> <p><img src="/media/Hands to feet for violin.png" title="An example SMILE resource from Charanga"/></p> <p>Each of the resources begins life as a <a href="">Sibelius</a> arrangement, from which we use both the midi and score output. We&#8217;ve got close to a thousand of these interactive pieces and with any job of this size, scriptable tools can really help speed up the production process.</p> <p>Pete, our musical arranger, asked me if there was a quicker way for him to generate the PNGs needed by the interactive tool. Up to now he&#8217;d been exporting them directly from Sibelius.</p> <p>Sibelius can batch export PDFs, so converting from these was definitely the way to go. The main problem with a straight <a href="">imagemagick</a> convert command, like:</p> <pre name="code" class="bash:nogutter:nocontrols"> convert score.pdf score.png </pre> <p><img src="/media/native.png" title="Converted with no options"/></p> <p>is that the native size of the <span class="caps">PDF</span> probably isn&#8217;t the correct size for the <span class="caps">PNG</span>, and when you try and force the correct size with a resize:</p> <pre name="code" class="bash:nogutter:nocontrols"> convert -resize 506x517 score.pdf score.png </pre> <p>You end up with a poor bitmapped image, because you might be scaling up from an effective smaller size; e.g. in my score example the native size of the <span class="caps">PDF</span> is only 271&#215;276 and I&#8217;m trying to go to twice the size [506&#215;517]. Hence the poor quality of the resulting <span class="caps">PNG</span>.</p> <p><img src="/media/blurry.png" title="Bitmap scaled up from a smaller PDF"/></p> <p>What&#8217;s required is to up the size of the <span class="caps">PDF</span> prior to the convert taking place; it is a vector format after all, so there&#8217;ll be no loss of quality with a larger image. A simple way to do this is to up the <span class="caps">DPI</span> of the <span class="caps">PDF</span>. ImageMagick will default to 72DPI unless told otherwise. Crank up the density (<span class="caps">DPI</span>) for a bigger image:</p> <pre name="code" class="bash:nogutter:nocontrols"> convert -density 600 -resize 506x517 score.pdf score.png </pre> <p>And the resulting <span class="caps">PNG</span> is much more acceptible:</p> <p><img src="/media/sharper.png" title="bitmap generated from a much larger, vector source image"/></p>