Jay Caines-Gooby Back in 5 mins 2011-04-04T14:39:05+01:00 Jay Caines-Gooby Wikileaks stats update 2010-12-08T00:22:25+00:00 2010-12-08T00:36:02+00:00 <p style="float: left; width: 280px; padding-top: 0; margin-top: 0"><img src="/media/311828080_15aaec2fb3_m.jpg" /><br /> <i><a href="">hourglass with wings</a>, by <a href="">lwr</a></i></p> <p>I&#8217;ve been <a href="">mirroring</a> <a href=""></a> since around 8am <span class="caps">GMT</span> Sunday (5th December). I was curious about the traffic.</p> <p>As of tonight (Tues 7th December, 23:50 <span class="caps">GMT</span>), some 64 hours later <a href=""></a> has served:</p> <ul> <li>2.7 GB of data</li> <li>13,804 unique visitors</li> <li>78,695 page views</li> <li>1,584 unique referrers</li> </ul> <p>I think I might need to upgrade its hosting plan&#8230;</p> Why I'm mirroring wikileaks 2010-12-06T16:39:19+00:00 2010-12-06T16:39:42+00:00 <p><img src='/media/wikileaks.png' style='float: left; padding-right: 20px; padding-bottom: 20px; height: 200px'/>I&#8217;m <a href="">mirroring wikileaks</a> as well as <a href="">providing <span class="caps">DNS</span></a> for it, because I believe that the heavy-handed attempts to censor it are the thin end of a very thick wedge.</p> <p>Let&#8217;s take a brief look at what&#8217;s happened recently, prior to the whole <a href="">wikileaks cablegate releases</a>:</p> <ul> <li>US Government <a href="">seizes the domain names of 82 websites</a>, without any due process</li> <li>UK coalition government <a href="">plans to try and end net neutrality</a> mirroring US moves</li> </ul> <p>If net neutrality ends, then broadcasters and news and entertainment corporations with deep pockets can pay to put their content in front of <span class="caps">ISP</span> users. There&#8217;s no incentive for the ISPs to prioritise or perhaps even carry packets to the second-tier domains. Thus we end up with lowest denominator content being the norm. Not that different from what we currently receive from our broadcasters. Obviously, this is a comfortable place for both the the media industry and government to be in. It&#8217;s the status quo. Channel 5, <span class="caps">ITV</span> 2, 3 &amp; 4, 24/7. No Wikipedia, no Vimeo, no blogs, and so on.</p> <p>If a government can both <a href="">lean on hosting</a> and <a href="">other internet companies</a> to stop providing their services to sites that the government would rather not see, and yet at the same time exercise the ability to remove a company from the internet by confiscating their domain, at what point will one threat follow another? Is it feasible that a US <span class="caps">ISP</span> who provides hosting to wikileaks, and who refuses to stop serving them, will then have their own domain confiscated?</p> <p>So, to me it seems that some heavy-handed governmental censorship is not far in the offing. Wikileaks is just the start and so I&#8217;m siding with the Internet on this one.</p> <p><a href="">Paul Carvill expresses this much more eloquently</a> than I can:</p> <blockquote> <p>&#8220;The free distribution of data, and resistance to top-down evaluation of the merit of that data, is what the web excels at. It is more important now than ever before that individuals are allowed to publish and consume information as they see fit, within the bounds of the law. The world wide web, must be allowed to operate neutrally and independently of governments and corporations, including domain name registrars, ISPs, data carriers and other and infrastructure providers. Everyone who uses the web benefits from such independence, and should promote and support it wherever possible.&#8221;</p> </blockquote> <p>The Tech Herald <a href="">has a detailed roundup of the wikileaks events this week</a>.</p> <div style="float: left"> <p>Twitter has played a big part in my involvement. On Friday I saw <a href="">Nic Ferrier</a> tweet:</p> <p><!--!/nicferrier/status/10727541568446464 --> <style type='text/css'>.bbpBox10727541568446464 {background:url( #9ae4e8;padding:20px;} p.bbpTweet{background:#fff;padding:10px 12px 10px 12px;margin:0;min-height:48px;color:#000;font-size:18px !important;line-height:22px;-moz-border-radius:5px;-webkit-border-radius:5px} p.bbpTweet span.metadata{display:block;width:100%;clear:both;margin-top:8px;padding-top:12px;height:40px;border-top:1px solid #fff;border-top:1px solid #e6e6e6} p.bbpTweet span.metadata{line-height:19px} p.bbpTweet span.metadata img{float:left;margin:0 7px 0 0px;width:38px;height:38px} p.bbpTweet a:hover{text-decoration:underline}p.bbpTweet span.timestamp{font-size:12px;display:block}</style> <div class='bbpBox10727541568446464'><p class='bbpTweet'>what about a campaign to get absolutely everyone to add wikileaks to their domain?<span class='timestamp'><a title='Fri Dec 03 16:10:19 +0000 2010' href='!/nicferrier/status/10727541568446464'>less than a minute ago</a> via web</span><span class='metadata'><span class='author'><a href=''><img src='' /></a><strong><a href=''>Nic Ferrier</a></strong><br/>nicferrier</span></span></p></div> <!-- end of tweet --></p> <p><a href="">Tom Morris</a> responded:</p> <p><!--!/tommorris/status/10755661272977408 --> <style type='text/css'>.bbpBox10755661272977408 {background:url( #efefff;padding:20px;} p.bbpTweet{background:#fff;padding:10px 12px 10px 12px;margin:0;min-height:48px;color:#000;font-size:18px !important;line-height:22px;-moz-border-radius:5px;-webkit-border-radius:5px} p.bbpTweet span.metadata{display:block;width:100%;clear:both;margin-top:8px;padding-top:12px;height:40px;border-top:1px solid #fff;border-top:1px solid #e6e6e6} p.bbpTweet span.metadata{line-height:19px} p.bbpTweet span.metadata img{float:left;margin:0 7px 0 0px;width:38px;height:38px} p.bbpTweet a:hover{text-decoration:underline}p.bbpTweet span.timestamp{font-size:12px;display:block}</style> <div class='bbpBox10755661272977408'><p class='bbpTweet'>@<a class="tweet-url username" href="" rel="nofollow">nicferrier</a> Zone file edited for That&#8217;s the sort of activism I can actually do. ;-)<span class='timestamp'><a title='Fri Dec 03 18:02:04 +0000 2010' href='!/tommorris/status/10755661272977408'>less than a minute ago</a> via <a href="" rel="nofollow">Echofon</a></span><span class='metadata'><span class='author'><a href=''><img src='' /></a><strong><a href=''>Tom Morris</a></strong><br/>tommorris</span></span></p></div> <!-- end of tweet --></p> <p>And then by Friday night, there were a ton of hosts and subdomains pointing at the changing wikileaks IP:</p> <p><!--!/wikileaks/status/10621245489938433 --> <style type='text/css'>.bbpBox10621245489938432 {background:url( #9AE4E8;padding:20px;} p.bbpTweet{background:#fff;padding:10px 12px 10px 12px;margin:0;min-height:48px;color:#000;font-size:18px !important;line-height:22px;-moz-border-radius:5px;-webkit-border-radius:5px} p.bbpTweet span.metadata{display:block;width:100%;clear:both;margin-top:8px;padding-top:12px;height:40px;border-top:1px solid #fff;border-top:1px solid #e6e6e6} p.bbpTweet span.metadata{line-height:19px} p.bbpTweet span.metadata img{float:left;margin:0 7px 0 0px;width:38px;height:38px} p.bbpTweet a:hover{text-decoration:underline}p.bbpTweet span.timestamp{font-size:12px;display:block}</style> <div class='bbpBox10621245489938432'><p class='bbpTweet'><span class="caps">WIKILEAKS</span>: Free speech has a number:<span class='timestamp'><a title='Fri Dec 03 09:07:56 +0000 2010' href='!/wikileaks/status/10621245489938433'>less than a minute ago</a> via web</span><span class='metadata'><span class='author'><a href=''><img src='' /></a><strong><a href=''>WikiLeaks</a></strong><br/>wikileaks</span></span></p></div> <!-- end of tweet --></p> <p>Saturday brought the mass-mirroring project:</p> <p><!--!/jerezim/status/11131990803750912 --> <style type='text/css'>.bbpBox11131990803750912 {background:url( #8f8f8f;padding:20px;} p.bbpTweet{background:#fff;padding:10px 12px 10px 12px;margin:0;min-height:48px;color:#000;font-size:18px !important;line-height:22px;-moz-border-radius:5px;-webkit-border-radius:5px} p.bbpTweet span.metadata{display:block;width:100%;clear:both;margin-top:8px;padding-top:12px;height:40px;border-top:1px solid #fff;border-top:1px solid #e6e6e6} p.bbpTweet span.metadata{line-height:19px} p.bbpTweet span.metadata img{float:left;margin:0 7px 0 0px;width:38px;height:38px} p.bbpTweet a:hover{text-decoration:underline}p.bbpTweet span.timestamp{font-size:12px;display:block}</style> <div class='bbpBox11131990803750912'><p class='bbpTweet'><a href="" title="#Wikileaks" class="tweet-url hashtag" rel="nofollow">#Wikileaks</a> mass-mirroring begins! Give server space to fight <a href="" title="#censorship" class="tweet-url hashtag" rel="nofollow">#censorship</a>! <a href="" title="#StreisandEffect" class="tweet-url hashtag" rel="nofollow">#StreisandEffect</a><span class='timestamp'><a title='Sat Dec 04 18:57:28 +0000 2010' href='!/jerezim/status/11131990803750912'>less than a minute ago</a> via <a href="" rel="nofollow">identica</a></span><span class='metadata'><span class='author'><a href=''><img src='' /></a><strong><a href=''>Jérémie Zimmermann</a></strong><br/>jerezim</span></span></p></div> <!-- end of tweet --></p> <p>Twitter has been <a href="">stalwartly been letting wikileaks news</a> through. At one point it seemed to be the only place to get the current IP of the official site.</p> </div> <p>There was an <a href="">Etherpad</a> being collaboratively edited on Friday, but that&#8217;s no longer available.</p> <p><a href="!/search/%23imwikileaks">#imwikileaks</a> and <a href="!/search/%23wikileaks">#cablegate</a> didn&#8217;t trend in the US or the UK across the weekend, but then again, <a href="!/search/%23xfactor">#xfactor</a> was on. There&#8217;s been a noticeable uplift in traffic this morning (Monday 6th), so we&#8217;ll see&#8230;</p> <h2>What a difference a weekend makes</h2> <p>In just three days, 3rd &#8211; 5th December, wikileaks went from a couple of official domains along with some unofficial mirrors that were being <span class="caps">DDOS</span>-ed off the web, to having <a href="">hundreds of working official mirrors</a> and a <a href="">full-text search of all the currently released documents</a>. Score one for the Internet.</p> <p>Still in doubt? <a href=""></a></p> Monit managing mongrels 2010-09-21T11:14:14+01:00 2010-09-21T11:15:02+01:00 <p><a href=""><img src="/media/monit.png"/></a></p> <p>Like <a href=";rls=en&amp;q=monit+mongrel_rails+execution+failed">everybody else</a>, I was getting the dreaded &#8220;Execution failed&#8221; from monit when it couldn&#8217;t restart a mongrel which had gone out-of-bounds on its monitoring settings.</p> <p>The solution that worked for me was the env and $<span class="caps">PATH</span> line:</p> <pre name="code" class="bash:nogutter:nocontrols"> start program = "/usr/bin/env PATH=/bin:/usr/local/bin:$PATH \ ruby mongrel_rails cluster::start \ -C /etc/mongrel_cluster/myapp.yml \ --clean --only 8000" stop program = "/usr/bin/env PATH=/bin:/usr/local/bin:$PATH \ ruby mongrel_rails cluster::stop \ -C /etc/mongrel_cluster/myapp.yml \ --only 8000" </pre> <p>But what scuppered me and will hopefully help you, is that whilst trying the various options suggested by Google, <em>I wasn&#8217;t reloading the monit config</em>. Doh!</p> <p>So if you&#8217;re trying to debug your failing mongrels under monit here are a couple of tips&#8230;</p> <p>Start a new shell, and unset your <span class="caps">PATH</span> to mimic how monit behaves:</p> <pre name="code" class="bash:nogutter:nocontrols"> sh unset PATH </pre> <p>Now you can test your monit stop/start lines (you&#8217;ll need to sudo as monit normally runs as root)&#8230;</p> <pre name="code" class="bash:nogutter:nocontrols"> /usr/bin/sudo /usr/bin/env PATH=/bin:/usr/local/bin:$PATH \ mongrel_rails cluster::stop \ -C /etc/mongrel_cluster/myapp.yml \ --only 8000 /usr/bin/sudo /usr/bin/env PATH=/bin:/usr/local/bin:$PATH \ mongrel_rails cluster::start \ -C /etc/mongrel_cluster/myapp.yml \ --clean \ --only 8000 </pre> <p>If that works for you, then you can be pretty sure that it&#8217;ll work for monit, as long as you <strong>remember to reload your monit config after making the changes</strong>.</p> Avoiding HTTPS mixed content warnings 2010-09-07T23:19:12+01:00 2010-09-08T08:24:21+01:00 <p><img src="/media/Screen shot 2010-09-07 at 22.37.19.png" title="Mixed HTTPS and HTTP content warning" alt="Skull and cross bones icon"/></p> <p>Working on an <span class="caps">HTTPS</span> site today I was getting mixed content warnings from Chrome due to a few external resources still coming over <span class="caps">HTTP</span>.</p> <p>Google analytics already has the <span class="caps">HTTP</span>/<span class="caps">HTTPS</span> switcher built into it, but jQuery was being pulled in with an explicit http call to:</p> <pre name="code" class="html:nogutter:nocontrols"> &lt;script src="" type="text/javascript"&gt;&lt;/script&gt; </pre> <p>I could serve the code directly myself, but it seemed neater to use the Google <span class="caps">CDN</span>. It was a relatively quick fix thanks to <a href="">stackoverflow</a>. I used the simplified version:</p> <pre name="code" class="html:nogutter:nocontrols"> &lt;script src="//" type="text/javascript"&gt;&lt;/script&gt; </pre> <p>which uses the // <a href="">protocol-relative <span class="caps">URL</span></a> to download the external jquery file. Because it&#8217;s served over <span class="caps">HTTPS</span>, you won&#8217;t benefit from using the browser&#8217;s cached version, but it is one less file to be served by you.</p> <h3>Don&#8217;t forget your favicon</h3> <p>So now I was at the point of being certain that every resource on the page was being served over <span class="caps">HTTPS</span>, yet I was still getting the dreaded mixed content warning. Then I realised I hadn&#8217;t explicitly put a favicon link in the html. A quick check in the logs seemed to confirm this. The implicit favicon.ico request was being made by Chrome, but using <span class="caps">HTTP</span>.</p> <p>Adding the icon link seemed to do the trick.</p> <pre name="code" class="html:nogutter:nocontrols"> &lt;link href='/images/favicon.ico' rel='shortcut icon' /&gt; </pre> <h3>Proxying Google maps</h3> <p>Once final problem was that as part of the registration process, I was showing a Google Map iframe.</p> <p>I didn&#8217;t want this page served over <span class="caps">HTTP</span> just to avoid the mixed content warning, especially as the map page contains personal details.</p> <p>As I&#8217;m using nginx to serve the site, and it&#8217;s relatively easy to proxy content served from local application servers like mongrel and unicorn, I wondered if we could do something similar with the requests to <a href=""></a>.</p> <p>The nginx config is really easy:</p> <pre name="code" class="nginx:nogutter:nocontrols"> location /maps { proxy_pass; proxy_set_header X-Real-IP $remote_addr; } </pre> <p>Just make sure this comes ahead of any location rule that matches /, as it needs more priority.</p> <p>And then in my code, I just invoke the iframe without the <a href=""></a> host, when I know I&#8217;m running in production, and hence running the proxy</p> <pre name="code" class="ruby:nogutter:nocontrols"> - host = (RAILS_ENV != "production" ? "" : "") %iframe#map{ :scrolling =&gt; "no", :marginheight =&gt; "0", :marginwidth =&gt; "0", :src =&gt; "#{host}/maps?hl=en&amp;amp;gl=GB&amp;amp;q=#{@address}&amp;amp;mrt=loc&amp;amp;t=m&amp;amp;z=15&amp;amp;iwloc=near;&amp;amp;output=embed", :frameborder =&gt; "0", :height =&gt; "250", :width =&gt; "250" } </pre> <p>and that, currently is doing the job for me.</p> Scripting hybrid CDROM & DVD burning with hdiutil and rake 2010-08-06T00:14:12+01:00 2010-08-06T01:48:36+01:00 <p><a href="">Max</a> was writing some rake tasks today and it reminded me to finish off this post which has sat unfinished for months.</p> <p>Bob&#8217;s been porting <a href="">Charanga&#8217;s music-teaching desktop software</a> from PC to Macs. The port is based on the work we&#8217;ve done in the past couple of years for our <a href=''>online products</a> and means that we can now have an online offering as well as PC and Mac desktop products, all built from the same codebase.</p> <p>We&#8217;ve <a href=";record_id=10180">initially released</a> <a href=";record_id=10181">three</a> <a href=";record_id=10182">products</a> with a further eight to follow.</p> <p>Each of these 11 products will come as either a hybrid <span class="caps">DVD</span> or <span class="caps">CDROM</span>, with both the PC and Mac version on, but only visible to relevant platform. Lots of CD burning products for Macs out there make it easy to burn these kind of ROMs, but the big problem is that they all need to be made manually. And with 11 different products, that&#8217;s 11 different manual processes, any one mistake of which could ruin the master that we&#8217;re sending off to the publisher.</p> <p>It occurred to me that we deploy <a href="">our</a> <a href="">web</a> <a href="">apps</a> with a single invocation:</p> <pre name="code" class="shell:nogutter:nocontrols"> cap deploy STAGE=production </pre> <p>So why not do the same with the burning of the CDROMs? Entirely automate the process so there&#8217;s no room for manual error&#8230;</p> <h3>Rake</h3> <p>Rake &#8211; <a href="">Ruby Make</a> &#8211; operates on a rakefile which defines lists of tasks, with optional requisite tasks that must first be completed. Given that building and burning the ROMs consists of a bunch of identical steps, differentiated only by the files that need to go on the relevant product&#8217;s <span class="caps">CDROM</span> or <span class="caps">DVD</span>, it sounds like an ideal tool, so let&#8217;s go ahead and build a skeleton rakefile&#8230;</p> <p>There are a bunch of files that are common to all the products, plus product specific files. These get pulled out of subversion (yes, yes, we&#8217;re only just migrating to git), copied into the product filestructure, the PC content gets added, the hybrid <span class="caps">ISO</span> image gets created and then we use this to physically burn the <span class="caps">ROM</span>.</p> <p>If we make each preceding step a prerequisite of the parent task, we can break the the steps down into nice self-contained pieces and have a single task invoke all the others below it.</p> <p>Ultimately, I wanted to be able to stick a <span class="caps">DVD</span> or <span class="caps">CDROM</span> into the drive and then call:</p> <pre name="code" class="shell:nogutter:nocontrols"> rake burn_electric_guitar_coach_dvd </pre> <p>And have a finished hybrid <span class="caps">DVD</span> pop out, fresh off the press.</p> <h3>Break it down</h3> <p>The tasks in the rakefile are roughly as follows:</p> <pre name="code" class="ruby"> # Define various constants # The source repository REPOSITORY_URL="" # Where we'll do all this stuff BUILD_DIR = File.expand_path "/Users/jay/Work/music coach" # The copy of the remote repository we'll use locally CACHED_COPY = "#{BUILD_DIR}/svn-cached-copy" # shared by all products COMMON_CONTENT = "'#{PRODUCTION_FOLDER}/help'" PRODUCTS = { # specific properties for each product :guitar_deluxe =&gt; { # Mac content :volume_name =&gt; "Play Electric Guitar", :app_folder_name =&gt; "Play Electric Guitar v3.0", :logo =&gt; "guitar deluxe.jpg", :modules =&gt; "First Lessons For Guitar", "Guitar Improver", "Guitar Songs And Styles", "Solo Guitar Performance Pieces", "Master Rock Power Chords", "Chord miner"] # PC content :pc_iso =&gt; "guitar_deluxe.cdr", :pc_iso_volume_name =&gt; "GuitarDeluxe", # details of which files to hide from a PC on a Mac and vice versa :hide_hfs =&gt; "{Common,Player,program files,Redist,System32,*.exe,*.inf,*.msi,*.ini}", :hide_joliet =&gt; "{.background,.DS_Store,.Trashes,,.fseventsd,Play Piano v3.0,Applications}" }, :electric_guitar_deluxe =&gt; { # ... }, :piano_deluxe =&gt; { # ... }, :play_piano =&gt; { # ... } # and so on for 7 other products } # A couple of helpers... # Input helper - gets input from user def ask message puts message STDIN.gets.chomp end # Symol helper - converts a string to a symbol # "Blah blah foo".symbolize = :blah_blah_foo class String def symbolize self.downcase.split(" ").join("_").to_sym end end # Now the tasks themselves # The default task (runs when rake is called without arguments) task :default =&gt; :create_repository # The create_repository task - builds a local copy of the repository for us to work from desc "Create a cached copy folder where the repository will reside which we can then svn export the installer files from" task :create_repository do # the production files unless File.exists?("#{CACHED_COPY}") puts "Creating initial cached copy of the repository" svn_user ||= ask("Enter your svn username: ") svn_password ||= ask("Enter your svn password: ") sh "svn checkout --username #{svn_user} --password #{svn_password} '#{REPOSITORY_URL}' '#{CACHED_COPY}'" end end desc "Update the cached copy of the respository to get latest versions of files" task :update_repository =&gt; [:create_repository] do puts "Updating #{topics_product} production files" sh "cd '#{CACHED_COPY}'; #{SVN_PATH}/svn update" end # Be DRY about the task creation and use some string to symbol magic to dynamically create the tasks # # This makes three tasks per product (11 products = 33 tasks :) # # # 1. build_#{topics_product}_dmg (with a prerequisite on 1.) # # 2. build_#{topics_product}_dvd (with a prerequisite on 2.) # # 3. burn_#{topics_product}_dvd (with a prerequisite on 3.) PRODUCTS.each do |topics_product, data| desc "Build #{topics_product} for Mac .dmg" task "build_#{topics_product}_dmg".symbolize =&gt; [:update_repository] do # We need to clean up .dmg and any old build folders # and make sure no other dmg of the same name is mounted sh "sudo umount -f '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}") sh "sudo rm -rf '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" if File.exists?("/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}") sh "rm '/tmp/#{PRODUCTS[topics_product][:dmg]}'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:dmg]}") sh "rm '/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg'" if File.exists?("/tmp/#{PRODUCTS[topics_product][:app_folder_name]}.dmg") # Take the read-only master .dmg that has the backgrounds, .DS_Store and folder stubs # and make a copy of it to /tmp, then resize the copy so we can add our content, then mount it sh "hdiutil convert '#{CACHED_COPY}/development/mac installer/#{PRODUCTS[topics_product][:dmg]}' -format UDRW -o '/tmp/#{PRODUCTS[topics_product][:dmg]}'" sh "hdiutil resize -size 4g '/tmp/#{PRODUCTS[topics_product][:dmg]}'; hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5" # The new, writable dmg is now mounted at '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}' # and it's where we'll assemble the rest of the dmg tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}/#{PRODUCTS[topics_product][:app_folder_name]}" # Sort the permissions out (99 is the magic OS X user and group that appears to be owned by the current user when viewed; i.e. my uid is 6, but when I ls -l a file owned 99:99 it appears as 6:6) sh "sudo chown -R 99:99 '#{tmp_product_dir}'" sh "sudo chmod -R 777 '#{tmp_product_dir}'" # Export the product modules to make this specific .dmg PRODUCTS[topics_product][:modules].each do |product_module| sh "cd '#{CACHED_COPY}'; #{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/#{product_module}' '#{tmp_product_dir}/modules/';" end # export the help sh "cd '#{CACHED_COPY}';#{SVN_PATH}/svn export --force '#{PRODUCTION_FOLDER}/help' '#{tmp_product_dir}/Production File System/help'" end # Right we're done making the .dmg so unmount it. The file will remain in /tmp sh "hdiutil detach -force '/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}'" end desc "Create the hybrid PC &amp; Mac .iso file for #{topics_product}" task "build_#{topics_product}_dvd".symbolize =&gt; "build_#{topics_product}".symbolize do # Create a Hybrid DVD image containing the contents of the PC .iso and the contents of the Mac .dmg # # get rid of any old tmp files sh "sudo rm -rf '/tmp/#{PRODUCTS[topics_product][:pc_iso_volume_name]}'" # mount the PC .iso sh "hdiutil attach '#{PRODUCTS[topics_product][:pc_iso]}'; sleep 5" # mount the Mac .dmg - this is where we'll copy the sh "hdiutil attach '/tmp/#{PRODUCTS[topics_product][:dmg]}'; sleep 5" tmp_product_dir = "/Volumes/#{PRODUCTS[topics_product][:app_folder_name]}" # Copy the contents of the PC .iso to the mounted Mac .dmg (which is why we resized it to 4Gb earlier) # hdiutil needs to operate on a mounted volume to successfully create a hybrid iso sh "ditto '/Volumes/#{PRODUCTS[topics_product][:pc_iso_volume_name]}' '#{tmp_product_dir}'" # - unmount the PC .iso sh "sudo umount -f /Volumes/'#{PRODUCTS[topics_product][:pc_iso_volume_name]}'" # Remove any previous hyrid .iso prior to making this one sh "rm -f '/tmp/hybrid.iso'" # Make the hybrid iso # exlude PC files from the Mac, and exclude Mac files from the PC sh "hdiutil makehybrid -o /tmp/hybrid.iso '#{tmp_product_dir}' \ -hfs -iso -joliet \ -hide-hfs '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_hfs]}' \ -hide-joliet '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}' \ -hide-iso '#{tmp_product_dir}/#{PRODUCTS[topics_product][:hide_joliet]}'" end desc "Burn the .iso file for #{topics_product} to DVD" task "burn_#{topics_product}_dvd".symbolize =&gt; "build_#{topics_product}_dvd".symbolize do # Burn the DVD # That really long device is the external burner's firmware address (we're using an external writer) # I can never remember how to get this. Use # # hdiutil burn -list | grep IOService # # to list your device sh "hdiutil burn -device IOService:/AppleACPIPlatformExpert/PCI0@0/AppleACPIPCI/PCIB@1E/IOPCI2PCIBridge/FRWR@3/AppleFWOHCI/IOFireWireController/IOFireWireDevice@d04b990c04c4d1/IOFireWireUnit/IOFireWireSBP2Target/IOFireWireSBP2LUN/com_apple_driver_Oxford_Semi_FW934_DSA/IOSCSIPeripheralDeviceNub/IOSCSIPeripheralDeviceType05/IODVDServices /tmp/hybrid.iso" end end </pre> <p>The end result of this is that we end up with 11 <code>burn_&lt;a product name&gt;_dvd</code> tasks, which each invoke in turn</p> <ol> <li><code>update_repository</code></li> <li><code>build_&lt;a product name&gt;_dmg</code></li> <li><code>build_&lt;a product name&gt;_dvd</code></li> </ol> <p>and ends up with a burnt, hybrid <span class="caps">DVD</span> being made for you.</p> <p>The critical part happens at line 163 where the Mac .dmg is mounted under /Volumes, and is writeable, has the PC content written to it. It seems that hdiutil only likes mounted images when creating hybrid images. I experimented with various other options (using directories in /tmp, etc) but for the hybrid image to built correctly, it seems this is your only choice. The -hide switches list via globs, which files to hide from each filesystem.</p> An /etc/init.d/unicorn script for multiple unicorn installations 2010-08-02T18:10:56+01:00 2010-08-06T00:29:39+01:00 <p>Our staging server needed to be able to run multiple <a href="">unicorns</a>, each responsible for a different rails app, e.g. QA and staging</p> <p>I wanted a simple /etc/init.d script that will start/stop/reload all my unicorns or just a specific one:</p> <pre name="code" class='bash:nogutter'> # starts all unicorns listed in /etc/unicorn/*.conf /etc/init.d/unicorn start # stops the QA unicorn /etc/init.d/unicorn stop /etc/unicorn/qa.conf </pre> <p>The /etc/unicorn files are just simple variable setters. Here&#8217;s a sample /etc/unicorn/staging.conf</p> <pre name="code" class='shell'> RAILS_ROOT=/var/www/apps/e_learning_staging/current RAILS_ENV=development </pre> <p>Here&#8217;s the script to <a href="">save as /etc/init.d/unicorn</a> (in case the gist doesn&#8217;t embed below) &#8211; don&#8217;t forget to run <code>sudo /usr/sbin/update-rc.d -f unicorn defaults</code> to link it up to your <a href="">rc.d scripts</a> for running at boot time.</p> <script src=""></script><p>Enjoy!</p> Realtime Election Tweets 2010-05-04T03:58:44+01:00 2010-05-17T09:56:34+01:00 <p><object width="576" height="400"><param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="always" /><param name="movie" value=";;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" /><embed src=";;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="576" height="400"></embed></object><p><a href="">Real-time UK General Election Tweets with node.js &amp; websockets</a> from <a href="">jaygooby</a> on <a href="">Vimeo</a>.</p></p> <p>Inspired by <a href="">Makato&#8217;s X-Factor real-time Twitter experiments</a> and with just a couple of days to go until the country goes to the polls, I wondered if we could glean an outcome from the Twittersphere, especially as we get nearer to the actual count and results.</p> <h2>Implementation</h2> <p>My <a href="">real-time election site</a>, monitors the <a href="">twitter streaming <span class="caps">API</span></a> for mentions of the <strike><strike>four</strike> three main British political parties, plus the Greens (hey, I live in Brighton Pavillion and they look like they might get their first seat) and their leaders, and tallies a total score and current ratio of tweets</strike>.</p> <p>I&#8217;ve since updated it and it monitors all the parties, plus various independents and niche (read 0 current seats in parliament) parties and also tallies phrases like &#8220;I voted for&#8221;, &#8220;I&#8217;m voting&#8221; etc to count actual votes as well as mentions.</p> <p>Using <a href="">node.js</a> and client-side <a href="">HTML5 websockets</a> (with a <a href="">fallback websocket implementation in Flash</a> for older browsers), the site is entirely implemented in Javascript. The server-side JS listens for Tweets and emits the scores as json via the client&#8217;s websocket. Some client-side JS handles the screen updates.</p> <h2>Server set-up</h2> <p>The static portions of the site are served by <a href="">nginx</a> (which is itself an asynchronous evented server like node.js). Initially I&#8217;d tried to serve the whole thing via node.js using the <a href="">paperboy.js static file serving module</a> but I&#8217;d need both paperboy and the <a href=""> websocket server</a> to share port 80, which would mean some re-engineering, and given that the election is in two days, wanted to get something up quickly!</p> <p>So my architecture is static files served out of docroot handled by nginx, and the websocket running on a high port for use by the browser clients.</p> <p>This high-port usage is itself a problem, as I&#8217;d imagine that many corporate firewalls block all bar 80 and maybe 8080, which is why the combined server running entirely out of node would be a good eventual goal.</p> <p>To try and solve this, I thought I&#8217;d try and proxy the high port via nginx. You can turn the <a href="">proxy buffer off</a> in nginx, making it ideal for this, but I hit another hurdle with the Flash implementation of the websocket that will be used by most browsers (only Chrome is currently able to use native HTML5 websockets).</p> <h3>Flash and the crossdomain policy file</h3> <p>Flash requests its cross-domain policy file in an <span class="caps">HTTP</span> call to us (aka our Socket Policy Server) thus:</p> <p><code> &lt;policy-file-request/&gt;\x00 </code></p> <p>Note there&#8217;s no <span class="caps">GET</span> verb, just the xml invocation and a <span class="caps">NULL</span> byte. Nice. Quite rightly nginx chokes on it, and issues an <span class="caps">HTTP</span> 400.</p> <p>There&#8217;s <a href="">an nginx hack</a> I could have used to get the socket policy served and perhaps correctly tunnel the websocket stream itself, but I just wanted to launch, so I&#8217;ve left this as a to-do for now.</p> Versioning your Amazon S3 buckets 2010-02-10T13:36:05+00:00 2010-02-10T13:39:56+00:00 <p><img src="/media/logo_aws.gif"/></p> <p>I&#8217;m sold <a href="">hook</a> <a href="">line</a> and <a href="">sinker</a> on the <span class="caps">AWS</span> platform. I&#8217;m especially impressed at the product innovation and ever-reducing prices.</p> <p>A few days ago Amazon announced <a href="">versioning for S3</a>. This means that with the versioning flag for a bucket switched on, you can retrieve earlier versions of your files. Sweet.</p> <p>Now, because I&#8217;m lazy, I tend to use <a href="">S3Fox</a> or <a href="">Cyberduck</a> for setting ACLs and creating european buckets and so on.</p> <p>Neither of these have updated yet to support the versioning flag, and the <a href=""><span class="caps">AWS</span> Console</a> doesn&#8217;t have an S3 interface, so I thought I&#8217;d get my hands dirty and find out how to do it with the <a href=""><span class="caps">REST</span> interface</a>.</p> <p>You issue a <span class="caps">PUT</span> to your bucket with the versioning querystring and the relevant <span class="caps">XML</span>:</p> <pre name="code" class="xml:nogutter"> &lt;VersioningConfiguration xmlns=""&gt; &lt;Status&gt;Enabled&lt;/Status&gt; &lt;/VersioningConfiguration&gt; </pre> <p>You can do it all with <a href="">cURL</a>, but the biggest pain is <a href="">generating the correct Authorization: header</a> to sign your <span class="caps">API</span> call.</p> <p>Enter Tim Kay&#8217;s <a href="">aws commandline tool</a> &#8211; its a swiss knife for S3 and EC2 calls. Enabling the versioning was as simple as:</p> <pre name="code" class="bash:nogutter"> aws put versioning.xml </pre> <p>Where versioning.xml contains the VersioningConfiguration xml snippet listed above.</p> <p>To check the versioning status of a bucket, you do:</p> <pre name="code" class="bash:nogutter"> aws get </pre> <p>Before you run the aws commandline, you&#8217;ll need to create an ~/.awssecret file with your <span class="caps">AWS</span> key and secret key. Don&#8217;t forget to chmod 600.</p> Sussex Internship Programme - usability and user-experience intern for Charanga 2009-12-04T14:29:14+00:00 2009-12-04T14:29:25+00:00 <p><img src="/media/sussex internships.gif" title="Sussex Internship Programme"/></p> <p><a href="">Charanga</a> are participating in the <a href="">2010 Sussex Internship programme</a>. We&#8217;re looking for a usability and user-experience intern to help us make our music e-learning system as easy to use as possible.</p> <p>The March 2010 internship positions should go live on Monday 7th December. Currently the <a href="">placements page</a> is only showing 40 positions, but the additional 60 places will be there from Monday. Then you&#8217;ll be able to <a href=";keyword=Charanga">search for our role</a>.</p> <p>Our instrumental, vocal and curriculum music e-learning system is used by 55 local authorities and thousands of teachers and students. You&#8217;ll be undertaking user testing, making recommmendations and working with our developers to implement some of these, gaining invaluable experience in the process.</p> A new, state-funded Montessori primary school for Brighton and Hove 2009-11-26T00:15:54+00:00 2009-11-26T00:17:02+00:00 <p><a href="">Thing 1</a> goes to the <a href="">Brighton &amp; Hove Montessori School</a>, courtesy of the government&#8217;s <a href="" title="EYSF">Early Years Funding Scheme</a> which we top up so she can attend for four mornings week and one full day.</p> <p>We love the <a href="">self-directed, independent, mixed age-group</a> learning environment it provides, but once she&#8217;s five, we&#8217;ll have to dig deep to fund her education.</p> <p>A group of Brighton &amp; Hove Montessori parents are campaigning for a state-funded Montessori primary school to be opened in the city. This builds on the precedent of five other state-finded primaries that have opened in the UK.</p> <p>At the moment, I&#8217;m only peripherally involved, but I did make this poster. The last time I did any print-work was using a pre-historic version of QuarkXPress, so this was an interesting challenge.</p> <p><a href="/media/Montessori Schools Foundation poster.pdf"><img style="width:570px" src="/media/Montessori Schools Foundation poster.jpg" title="Poster to support a new state-finded Montessori school in Brighton & Hove"/></a></p> <p>If you&#8217;ve got a child aged 3-11 and you&#8217;re concerned about or dissatisfied with the increasingly restricted range of schooling choices for your child in Brighton &amp; Hove, then we&#8217;d <a href="">welcome</a> <a href="">your</a> <a href="">support</a></p>