A clean build of the "generic" Android platform on an Intel Core 2 Duo machine with 3GB of RAM running Ubuntu 9.10 32-bit generally takes around 1.5 hrs. I know this because I have built Android many many times. That's some solid sword fighting time.
Fun? Yes! Productive? Nay! I want fast Android builds! With a modest budget and an eye for speed, we should be able to significantly improve our build times. Since we are using the standard GNU make utility for our builds, we can take advantage of parallelization by running as many jobs as we have processing threads. The repo sync command, albeit network dependent, can also be parallelized with the -j flag to fetch multiple projects at once. Having enough RAM to keep everything in memory during the linking phase is helpful. Of course a faster hard drive or a solid state drive definitely helps speed up the I/O.
Luckily, I had purchased a HP Pavilion Elite HPE-150f a few months ago. The specs for this desktop are as follows:
And there you have it! A $1,200 CDN machine can build Android in 16 minutes! The Intel Core i7 has 8 processing threads, so the commands I used were repo sync -j9 and make -j9. Generally 1 + number of processing threads works the best when deciding how many jobs to run. Sadly the sync, being network bound, still takes ~30mins. You rarely need to do a clean sync so I can live with that.
There is one tragic consequence of these 16 minute builds: your sword fighting technique will suffer terribly. This could cost you dearly if ever you have to cross swords with d'Artagnan, Porthos, Athos or Aramis.
Fun? Yes! Productive? Nay! I want fast Android builds! With a modest budget and an eye for speed, we should be able to significantly improve our build times. Since we are using the standard GNU make utility for our builds, we can take advantage of parallelization by running as many jobs as we have processing threads. The repo sync command, albeit network dependent, can also be parallelized with the -j flag to fetch multiple projects at once. Having enough RAM to keep everything in memory during the linking phase is helpful. Of course a faster hard drive or a solid state drive definitely helps speed up the I/O.
Luckily, I had purchased a HP Pavilion Elite HPE-150f a few months ago. The specs for this desktop are as follows:
- Intel Core i7-860 Processor (2.8 GHz, 1MB L2 + 8MB Shared L3 Cache)
- 8GB PC3-10600 DDR3 SDRAM memory (4x2048MB for ultimate performance) (expandable to 16GB)
- 1 Terabyte (TB) Serial ATA hard drive running at 7200 RPM
- Windows 7 Premium 64-bit
Hmmm... I wonder how my Android build will perform on this machine? Okay, dual boot time. So I install the latest Ubuntu 10.04 LTS (Lucid Lynx) 64-bit and grab all the required packages to build Android.
Next, I set up a buildbot for Android so we can build in style. Note that because of Android's repo tool, buildbot doesn't work out of the box; you have to code in support for repo. Okay, now that's working let's give it a shot!
And there you have it! A $1,200 CDN machine can build Android in 16 minutes! The Intel Core i7 has 8 processing threads, so the commands I used were repo sync -j9 and make -j9. Generally 1 + number of processing threads works the best when deciding how many jobs to run. Sadly the sync, being network bound, still takes ~30mins. You rarely need to do a clean sync so I can live with that.
There is one tragic consequence of these 16 minute builds: your sword fighting technique will suffer terribly. This could cost you dearly if ever you have to cross swords with d'Artagnan, Porthos, Athos or Aramis.
You can add a Solid-State drive, EXT4 or BTRFS.. that should give you another impressive boost! :)
ReplyDeleteHi
ReplyDeletecan you help me with the buildbot config, due to don't know how to config this for repo scrip.
Download the buildbot source code and look how they added support for git. For basic operations like checkout and update it is fairly simple to add a repo command handler to buildbot. You can also look at how Chromium added support for gclient in their buildbots. All that code is open source.
ReplyDeletePierre - you can speed up new syncs too by setting up a mirror repo and then using that as a reference.
ReplyDeleteTry using the repo init --reference command (it is in the latest version of repo)
Thanks naseer! Great tip!
ReplyDelete