TIP: Compile your programs fasters with multiple processor machines
If you have a dual core or a multiple processor machine then you can compile your programs much faster than on a single processor machine. I will show how you can compile your a Linux kernel on your Debian machine twice as fast by setting the concurrency level.
For example, in one of my previous post we saw how to measure the amount of time a particular program takes to execute. In that example, we saw it took around 10 minutes to compile a custom 2.6.28 kernel on a AMD64 Anthlon Dual core machine:
# time make-kpkg –append-to-version=-test kernel_image
real 10m58.707s
user 9m38.700s
sys 1m12.013s
However in the above example, I was not efficiently using the power of both the processors. I simply gave the standard “make-kpkg” command to compile the kernel.
Now let’s see how we can take advantage of our dual-core machines and save us some valuable time.
# set CONCURRENCY_LEVEL=2
# export CONCURRENCY_LEVEL=2
# time make-kpkg –append-to-version=-test kernel_image
and here comes the best part – the results:
real 5m28.433s
user 4m18.320s
sys 1m13.276s
As we can see from the results above, we were able to compile the same kernel on the same machine in half of the time – approximately 5 minutes.
How do we find out what CONCURRENCY_LEVEL number?
Some people say that you should set the level to: Number of processors in your machine+ 1 and some say that you should set it to: Number of processors in your machine. I personally use the later.
To find out the number of processors in your machine, give the following command:
# grep -c ^processor /proc/cpuinfo
2
and then you can set the concurrency level to whatever is the output of the above command. In my case it is “2″.
Now what if you are use to compile your programs/Linux kernel with the classic “make” command and you would like to make use of the concurrency level. In that case you can simple use the “-j” flag like this:
# make -j2 bzImage
Hope this will help you to save some time.

Free Email Subscription









May 8th, 2009 at 2:23 pm
For some reason, using make-kpkg, this doesn’t work for me. The “grep -c ^processor /proc/cpuinfo” yields 4 processors, and top shows, 25% utilized regardless of whether I have “export CONCURRENCY_LEVEL=4″. Not sure what I could do differently…
Reply to this comment
May 8th, 2009 at 6:32 pm
Never mind, I figured it out. I was sudoing and I needed to sudo -s so that the export was in effect for the root user.
Reply to this comment
January 24th, 2010 at 12:23 pm
Thanks for the tip! This is great information. However, the CONCURRENCY_LEVEL env var did not work for me. Only the -j switch to make made a difference. I was running both as “root” user.
Reply to this comment
Admin Reply:
February 16th, 2010 at 8:00 am
The CONCURRENCY_LEVEL=2 should have made the difference. It is same as using “-j2″. Did you timed your compilation? What distribution are you using? I think this is only supported in Debian/Ubuntu. For non Debian based distros you can use “j” flag…
Reply to this comment