Building Android in 16 minutes!

A clean build of the "generic" Android platform on an Intel Core 2 Duo machine with 3GB of RAM running Ubuntu 9.10 32-bit generally takes around 1.5 hrs. I know this because I have built Android many many times. That's some solid sword fighting time.



Fun? Yes! Productive? Nay! I want fast Android builds! With a modest budget and an eye for speed, we should be able to significantly improve our build times. Since we are using the standard GNU make utility for our builds, we can take advantage of parallelization by running as many jobs as we have processing threads. The repo sync command, albeit network dependent, can also be parallelized with the -j flag to fetch multiple projects at once. Having enough RAM to keep everything in memory during the linking phase is helpful. Of course a faster hard drive or a solid state drive definitely helps speed up the I/O.
Luckily, I had purchased a HP Pavilion Elite HPE-150f a few months ago. The specs for this desktop are as follows:

• Intel Core i7-860 Processor (2.8 GHz, 1MB L2 + 8MB Shared L3 Cache)
• 8GB PC3-10600 DDR3 SDRAM memory (4x2048MB for ultimate performance) (expandable to 16GB)
• 1 Terabyte (TB) Serial ATA hard drive running at 7200 RPM
• Windows 7 Premium 64-bit

Hmmm... I wonder how my Android build will perform on this machine? Okay, dual boot time. So I install the latest Ubuntu 10.04 LTS (Lucid Lynx) 64-bit and grab all the required packages to build Android.

Next, I set up a buildbot for Android so we can build in style. Note that because of Android's repo tool, buildbot doesn't work out of the box; you have to code in support for repo. Okay, now that's working let's give it a shot!

And there you have it! A $1,200 CDN machine can build Android in 16 minutes! The Intel Core i7 has 8 processing threads, so the commands I used were repo sync -j9 and make -j9. Generally 1 + number of processing threads works the best when deciding how many jobs to run. Sadly the sync, being network bound, still takes ~30mins. You rarely need to do a clean sync so I can live with that.
There is one tragic consequence of these 16 minute builds: your sword fighting technique will suffer terribly. This could cost you dearly if ever you have to cross swords with d'Artagnan, Porthos, Athos or Aramis.

Sorting Photos by Filename on the iPad #fail

For a company that is known for designing great user interfaces that are pleasant and simple to use, Apple definitely made it hard for me to do something trivial. So I have a folder of image files with numerical names on my Windows PC, e.g:

00.png

01.png

02.png

...

I want to transfer them to my iPad so I can enjoy them on the go. I pull out my trusty Apple 30 pin to USB cable and connect my PC to my iPad and wait for iTunes to launch. At this point I'm thinking to myself, on my Android phone I could do all this over the air... Sigh. But let's continue... I sync my folder to the iPad and disconnect.

At this point I expected to have my photos in the logical order defined by my clever numeric nomenclature, but nay! I seem to have an arbitrary ordering of my photos. WTF?

So how do I solve this problem? ...Google... According to the plethora of Apple fanboys out there on the Internets, I should switch to a Mac and use iPhoto which has an option to sort by filename. That's great! But I'm a PC :( --Any other bright ideas?

Well, apparently, according to Apple Support (http://support.apple.com/kb/HT4221):

If your iPhone, iPad, or iPod touch is using the latest software, it sorts photos by the Date Taken tag of the photos. Specifically, your device uses the pictures' EXIF tags to determine the Date Taken information. Your device will look for the following EXIF tags within your pictures:

1. Capture Date
2. Date Time Digitized
3. Date Time Original
4. Last Modified

Okay! I'll work around the iPad's UI deficiency and modify the Date Taken of all my photos. So I fire up Windows Live Photo Gallery, select all files in that folder, change the Date taken to Aug 2nd 2010 at 11:00am. Awesome! Now I reconnect my iPad to the PC, delete the incorrectly sorted folder and resync with the newly updated folder... No dice. Same ordering as before.

Maybe it's because I changed all the modification times to the same date? Maybe iTunes caches the photos? I don't know what the issue is. I'll assume it's the former. Okay I'll resort to python now.

...code...code...code...

Here we go: sort_ipad_photos.py

#!/usr/bin/python

import datetime, time, sys, os

# iPad sorts photos by timestamp rather than by name. A solution to this is to

# change the timestamps for files so that they are sorted in the same order as

# the file names.

def get_next_time(count):

future = datetime.datetime(2010, 1, 1, 0, 0)

future += datetime.timedelta(minutes = count)

timestamp = time.mktime(future.timetuple())

return timestamp

def main():

folder = sys.argv[1]

list = os.listdir(folder)

list.sort(reverse=True)

count = 0

for file in list:

if file.lower().endswith('jpg') or file.lower().endswith('png'):

count += 1

time = get_next_time(count)

print('Changing timestamp of %s to %d' % (file, time))

os.utime(file, (time, time))

USAGE = 'Usage: python sort_ipad_photos.py folder'

if __name__ == '__main__':

if len(sys.argv) != 2:

sys.exit(USAGE)

main()

So this script will essentially change the modification and access timestamps of the files so that the first image will be the most recent one. E.g. if there are 20 images, #20 has a timestamp of Jan 01 2010 00:01 and #19 has a timestamp of Jan 01 2010 00:02. One minute increments.

This should solve my problem. I run the script and test out the ordering with `ls -t`. Looks good. I repeat the delete and sync folder to my iPad process again. No dice. Same ordering! WTF?!

The issue must be the creation date. I modified the access and modification times but not the creation date. New solution: Create new files sequentially so that their timestamps have the same order as their filenames.

...code...code...code...

#!/usr/bin/python

import time, sys, os, shutil

# iPad sorts photos by timestamp rather than by name. A solution is to

# create new files with creation dates that match the ordering of the filenames.

def create_new_files():

folder = sys.argv[1]

list = os.listdir(folder)

list.sort(reverse=True)

sorted_folder = folder + '/sorted'

if not os.path.exists(sorted_folder):

os.mkdir(sorted_folder)

count = 0

for file in list:

if file.lower().endswith('jpg') or file.lower().endswith('png'):

shutil.copyfile(file, sorted_folder + '/' + file)

time.sleep(1)

def main():

create_new_files()

USAGE = 'Usage: python sort_ipad_photos.py folder'

if __name__ == '__main__':

if len(sys.argv) == 1:

sys.exit(USAGE)

main()

Done! Okay this one actually worked! Thank you Apple for wasting 2 hours of my time!