Kushal Koolwal’s Debian Linux Blogs

Purpose:
Sometimes if you are experimenting with Linux on different machines, there are chances that upon switching to a different machine your Ethernet card won’t be activated by default i.e. upon giving the command ‘ifconfig’ your Ethernet card will disappear even if your driver is loaded properly.

Test Setting:
Linux O.S.: Debian 4.0
Kernel: 2.6.22 - 2.6.24

Scenario:
For example, you install your Linux system on one machine and then you take your hard disk and attach it to an another machine which has possibly the same configuration. On booting you find that you cannot see your eth0 or eth1 upon giving ‘ifconfig’. For example, you see this

debian:~# ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:560 (560.0 b) TX bytes:560 (560.0 b)

instead of this

debian:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:01:02:03:04:05
inet addr:192.168.0.101 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe81::202:244:fe68:3172/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:78108 errors:0 dropped:0 overruns:44 frame:0
TX packets:50313 errors:0 dropped:0 overruns:0 carrier:0
collisions:2 txqueuelen:1000
RX bytes:103308543 (98.5 MiB) TX bytes:3763441 (3.5 MiB)
Interrupt:169 Base address:0xe000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:560 (560.0 b) TX bytes:560 (560.0 b)

However, you can activate your Ethernet card by giving commands like:

# dhclient eth2

or

# dhclient eth3 (If you have Dual Ethernet card)

Suppose now you can attach your same hard drive to another identical machine and now you again cannot activate your Ethernet card even by giving the above commands i.e. ‘dhclient eth2′ or ‘dhclient eth3′. But you can activate it now by giving commands like:

# dhclient eth4

or

# dhclient eth5 (If you have Dual Ethernet card)

This cycle can go on and on i.e. upon switching machines every time you have to keep incrementing the eth<number> by one or by two (if you have dual Ethernet card).

If you observe something like this on your machines then read the solution below.

Note: Before you proceed to the solution make sure that your Ethernet card driver is loaded. If your driver is compiled as a module in your kernel, you can load it by giving the following command:

# modprobe <name-of-drive>

For example,

#modprobe e100

If it is built into the kernel then you don’t need to give the command as it should have got loaded automatically. Steps to configure and compile your Ethernet driver is out of the scope of this post.

Solution:
Remove the following two files from the udev directory:

debian:# cd /etc/udev/rules.d

debian:# rm z25_persistent-net.rules z45_persistent-net-generator.rules

This happens because of the above two configuration files that udev creates by default. The file ‘z25_persistent-net.rules’ creates rules for the Network devices and remembers them upon every boot. Removing this files forces the udev to create names from start i.e. eth0 upon every reboot. I don’t claim that I understand this issue completely but if you want to go to the depth of it then I recommend reading the udev documentation under the directory ‘/usr/share/doc/udev’. It has vast source of information. Also you can read more about udev in the Kernel Documentation directory under ‘/usr/src/linux/Documentation’.

I noticed this behavior in Debian 4.0 (Stable/Etch). This issue was not there in Debian 3.1 (Stable/Sarge). This means that something changed in udev package between 3.1 and 4.0. Go figure!

I was able to figure out this solution by reading the documentation mentioned above. Also this lead me to create a post which explains what steps one should take in general to (re)solve problems under Linux.

As usual, please leave a comment/feedback, if you have any.

Purpose:

Suppose you have multiple machines/computers running Debian Linux and you do experiment with them a lot like installing various .deb packages, tweaking some hardware, installing kernels, testing various hardware and then re-installing the system again and the cycle goes on and on. Now the problem with this is that every time you do a re-install you have to download the packages again on each machine.

For example just to get a basic graphical environment running you need to do the following at least:

# apt-get install gnome gdm xserver-xorg xfonts-base

This itself is about 237 MB of downloads based on Debian Etch (4.0) as on 15th April 2008.

Now imagine if you have to do this on each of your five network machines. Won’t it p so much painful to download them again and again, especially if you have a slow Internet connection? However there is a Debian utility called “apt-proxy” by which you just have to install the packages once on a “Host” computer and then the rest of your other computers i.e. “Clients” will download from the Host computer’s hard drive rather from Debian Repository over the Internet.

Difference Proxy Cache Server versus Debian Mirror?

Actually many people (just like me) want to setup Proxy Cache servers but they confuse the terms (or are not aware of) with setting up Debian Mirror. Setting up a Debian Mirror is time and resource consuming activity and you need a dedicated server to do that and at least a 500 GB of hard drive just to store the deb packages.

For more details please refer to the following links:

1. http://www.debian.org/mirror/ftpmirror

2. Debian APT HOW-TO Manual

If you came here looking for how to setup a Debian Mirror then I am afraid that this article is not for you.

How do I setup a cache proxy server?

In this article we will use the following terminology:

Host Computer - Which will act as a Debian local repository

Client Computers - Will download packages from the Host computer

Step 1: Install apt-proxy package

Log in to any one your Debian Linux computer on your network and do the following:

# apt-get udpate

# apt-get install apt-proxy

Note: We called this computer a Host computer because we have installed apt-proxy on it. In short, any computer on which you install apt-proxy package becomes the Host computer. You only need to install it on one computer on your network. Also this Host computer should have sufficient amount of disk space to store the packages. Any normal user won’t install more than 10GB of packages but again it depends. This is something that you will have to determine on your own.

You DO NOT need to install apt-proxy package on any of your client computers.

Step2: Configure the apt-proxy-v2.conf file

Now on your host computer edit the apt-proxy-v2.conf file:

# nano /etc/apt-proxy/apt-proxy-v2.conf

This file contains various variables that you can configure. In most of the cases you won’t be required to make any changes to the file. The default settings will just work fine.

However make sure that you have following lines un-commented (if they are not):

;; Server port to listen on
port = 9999

;; Cache directory for apt-proxy
cache_dir = /var/cache/apt-proxy

[debian]
;; The main Debian archive
;; You can override the default timeout like this:
timeout = 30

;; Backend servers, in order of preference
backends =
http://ftp.us.debian.org/debian

[security]
;; Debian security archive
backends =
http://security.debian.org/debian-security

To un-comment a line means to remove the symbol semi-colon(”;”) from the beginning of the line. For example,

;port = 9999 - This is a comment.

port = 9999 - This is a un-commented line.

You can download my sample apt-proxy configuration file and check against yours in case if you are not sure what is going on.

Once you do that, all you need to do is to restart your apt-proxy daemon like this:

# /etc/init.d/apt-proxy restart

This will take your new settings which you did in the apt-proxy-v2.conf file into effect. This is an IMPORTANT step.

Step 3: Get the IP address of your Host Computer

Before we move ahead to configure the Client computers we need to get the IP address of the Host computer.

You can easily get this information from ifconfig command.

Let’s say that the IP address of your Host computer is 192.168.0.10

We will use the above IP address for the rest of the article. You can substitute with your own IP address of the Host computer.

Step 4: Configure sources.lst file on your Clients

Now on each of your Client computers, edit the sources.lst file to reflect your new Host computer’s IP address that we found in Step 3:

# nano /etc/apt/sources.lst

And change the following lines:

#Stable

deb http://mirrors.kernel.org/debian stable main contrib
deb-src http://mirrors.kernel.org/debian stable main contrib

to

#Stable
deb http://192.168.0.10:9999/debian stable main contrib
deb-src http://192.168.0.10:9999/debian stable main contrib

And similarly do for your Security repositories:

#For Security patches
deb http://mirrors.kernel.org/security stable/updates main contrib

to

#For Security patches
deb http://192.168.0.10:9999/security stable/updates main contrib

There is a high chance that your original sources.lst file may have some other mirror like the following:

deb http://http.us.debian.org/debian/ stable main contrib non-free

In that case too all you need to do is to change the http.us.debian.org part to 192.168.0.10:9999/

Step 5: Ready to go

Now just give the command:

# apt-get update

on all of your client computers on which you modified the sources.lst file as explained above. This will result in fetching of the deb packages from your Host computer from now onwards.

Additional Notes/FAQ:

Q:How does the Host computer have the deb package that client is requesting?

A: If you did install a particular package on one of your Client computer after you restarted your apt-proxy daemon on your Host computer then your Client computer will download the package from your Host computer.

For example, you do following at your Client computer:

# apt-get install traceroute

Your Host computer will get the package from the Internet (Debian Repository) just like the default (normal) way the Debian system works. Your Host computer will fetch packages in the same way as it use to do before installing apt-proxy. The only difference is that your Host computer now will store (cache) a copy the ‘traceroute’ package in ‘/var/cache/apt-proxy’ directory also.

And then suppose you do the following on your 2nd Client computer:

# apt-get install traceroute

Now your Client computer will fetch the package from your Host computer, specifically from the directory ‘/var/cache/apt-proxy’ which we configured in Step 2 above.

You can actually see the packages that are now being “cached” in the ‘/var/cache/apt-proxy’ directory of your Host computer over time as and when you install more and more packages on your Host/Client computers:

# ls /var/cache/apt-proxy/debian/pool/main/

a/ e/ i/ libb/ libf/ libm/ libr/ libw/ o/ t/ x/
b/ f/ k/ libc/ libg/ libn/ libs/ libx/ p/ u/ y/
c/ g/ l/ libd/ libi/ libo/ libt/ m/ r/ v/ z/
d/ h/ liba/ libe/ libj/ libp/ libv/ n/ s/ w/

The above directory structure is just similar to what we expect to find on Debian repository and hence apt-get from your Client machines can easily fetch them.

Q: What if the package that we request from Client machine is not present on Host machine?

A: Certainly a good question. In that case your Client computer will still request the package from the Host computer and the Host computer will first check in it’s cache i.e. ‘/var/cache/apt-proxy/’ directory for the package requested. Upon not finding, it will download from Internet (Debian Repository) and your Client will then fetch from the Host. This process is almost transparent to the Client user. After this your Host computer will cache the package for future requests from other Clients and will not have to download from Internet.

Q: What if I installed a package on Host computer, will that get cached too?

A: At first I thought it should do, but I tested out on my system and it seems that it does cache it. But I am sure there might be a way to do that. Read the documentation and man pages for this. Update: Actually you can do this. You need to use the tool ‘apt-proxy-import’ and you can import your entire apt-get cache directory contents (debs) into the apt-proxy cache. The apt-get cache’s default location is ‘/var/cache/apt/archives/’.

Q: There is something called apt-cacher also which does the same. What is the difference between apt-cacher and apt-proxy?

Q: Some of my clients uses stable repository and other uses testing/unstable repository. Will apt-proxy still work?

A: Yes, it will. It will basically cache any package that your client requests (if the repository is enabled in the apt-proxy-v2.conf file). Also, it does not matter if your Host computer uses stable/testing/unstable, it will still function properly. You can also use combination of Debian and Ubuntu repository by un-commenting the proper lines in the apt-proxy-v2.conf file.

A: Yes. apt-cacher and apt-proxy do the same job. The major difference is that in apt-cacher you don’t need to change the sources.lst file on your Client computers. All you need to do is to create/change a file at ‘/etc/apt/apt.conf.d/01proxy’. See more on this page.

Known Issues:

1. If you change the default location of cache directory from /var/cache/apt-proxy to something else like ‘/root/proxy/cache’ in apt-proxy-v2.conf file and restart the daemon, it won’t work. At least it my case, I was not able to get it working. So I resorted to the default location. May be it is a permission issue and we may need to give apt-proxy permission to write in ‘/root/’ directory.

Keep tuned-in: More on this topic in my future posts.

Recently I was going through my Newsforge feeds and I noticed an excellent article about a utility called “fio”. It is a disk performance checking utility - a better alternative to the good old “hdparm” utility.

My only issue was that there was no native package for the Debian repository. And I wanted to install one on my Debian (Lenny/testing) system. Therefore I decided to create a “.deb” package for the fio using checkinstall. Note that I had to use “checkinstall” instead of “debuild” because fio is not a native Debian package as I mentioned before. So I had to download the fio source and then create a Debian Binary package for fio on my Debian system.

I thought that people who also want to install fio on their Debian system can utilize the package that I created. This will save them the trouble of going through the process of making binary package although I have to say that it is not an overly complicated process.

You can download the file here.

Also I high recommend you to read README and HOWTO file that comes with the source. They are very comprehensive - very few ones are that great.

Update: Based on a feedback from one of the users, it seems that you will need to install libaio1 package (shared library) before you can install the FIO package. You can do this by:

# apt-get install libaio1

and then

# dpkg -i fio_1.19-etch-build-1_i386.deb

Enjoy the utility!

As usual, please leave a comment/feedback, if you have any.

Purpose:

This blog entry explains how you can determine whether your Serial Ports on your system works or not under Debian Linux.

Requirements:

Two computers running Debian Linux

At least one Serial Port on each of the computer

A Serial Cable that runs from one computer (Computer A) to another computer (Computer B) with proper gender at both ends.

Serial Port driver available in the Kernel. Almost all the kernels have it configured in them.

Software: minicom & setserial (if required)

Install the software by giving commands:

apt-get update

apt-get minicom setserial

Basics:

Before we begin, it is important to know that your Serial Ports (COM1 and COM2) are named as /dev/ttyS0 and /dev/ttyS1 respectively and so on.

BIOS configuration: It is important to first check your BIOS and see if Serial Ports are disabled or not. If they are disabled then please enable them and assign them proper IRQ before you proceed.

The BIOS default settings are as follow:

COM1 - 0×3F8 = IRQ4

COM2 - 0×2F8 = IRQ3

Almost all the BIOSes have the above setting. Therefore, there are very less chances that you will have to modify those in the BIOS.

Tell me how to test now?

Step 1: Connect both the serial port of computer A to serial port of computer B using the Serial Cable.

Step 2: Check if Linux detects your Serial Port or not.

There are many ways to do that. One way you could do is to give the following command:

debian:~# dmesg | grep ttyS

You should see something like this:

serial8250: ttyS0 at I/O 0×3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0×2f8 (irq = 3) is a 16550A
00:09: ttyS0 at I/O 0×3f8 (irq = 4) is a 16550A
00:0a: ttyS1 at I/O 0×2f8 (irq = 3) is a 16550A
debian:~#

The above output shows that your COM1 and COM2 ports are available and Linux recognizes them.

Another way to test is to give the following command:

debian:~# ls > /dev/tyS0

If the command does not return any error message then your Serial Ports are recognized by your Linux system.

Errors?

In case, Linux does not detect your Serial Port, then try giving the following command:

setserial /dev/ttyS0 autoconfig auto_irq

Substitute ttyS0 with ttyS1 as required.

Step 3: Configure your Serial Ports.

Launch the minicom program from your terminal and input the following settings after navigating through the minicom menu (Ctrl A -Z):

Select the device: ttyS0 or ttyS1 (Depending on whatever device name is being assigned to your serial port)
Buad Rate: 57600
Bits per second: 8
Parity: None
Stop Bits: 1
Hardware Flow Control: Yes
Software Flow Control: No

Here is an example of the settings:

Once you configure your Serial Port with above parameters, you are ready to start communicating between the two computers through Serial Connection.

Step 4: Start communicating.

Before you can start communicating, you need to perform the Steps 2 and 3 on your other computer also. Once you do that you should be in a state where you have minicom window in front of you on both the computers.

Now start typing some characters from computer A (in your minicom window typing screen) and you should see those characters appearing auto-magically on your computer B. For example like this:

Errors?

In some case, you might get an error as follow as soon as you try to launch “minicom” program:

minicom
minicom: WARNING: configuration file not found, using defaults
Device /dev/modem access failed: No such file or directory.

All this means is that you need to pass on the flag -s when you launch minicom like this:

minicom -s

Once you do that you should be able to select your serial port device and other parameters.

That’s it! You have successfully detected and tested Serial Ports on Debian Linux.

If you are further interested in this topic, then I high recommend you to read the following articles:

1. http://www.cpqlinux.com/serialconsole.html

2. http://www.vanemery.com/Linux/Serial/serial-console.html

As usual, please leave a comment/feedback, if you have any.

Purpose:

If you have been using Linux for over two to three years than chances are that you may have been in a situation where you need to modify your Kernel’s Makeconfig or Kconfig files or adding a new device driver for your hardware from your hardware vendor which is not yet into the mainstream kernel (www.kernel.org).

Once you do all of the above then you may need to generate a patch (for the changes that you just made) so that other users can simply apply that patch rather than going into the kernel sources and manually modify all those files. This blog posting will explain in a simple way how you can do that. Although I think there might be several postings which will explain you how to do that but they may not be clear or may not cover your case specifically.

Let’s start with an example:

I downloaded Kernel source 2.6.24 into my /usr/src/linux directory. Suppose I need to add a framebuffer driver support, say for example, for my VIA’s Unichrome Graphic chipset. I copy the framebuffer driver source directory (provided by VIA) into my kernel directory as follow:

cd /usr/src/linux

cp -a /usr/src/fbdev-via-unichrome /usr/src/linux/driver/video/via

Note: The directory /usr/src/linux/driver/video/via did not existed when we installed 2.6.24 kernel source initally.

After this suppose we need to make changes to files Kconfig and Makefile in /usr/src/linux/drivers/video directory.

Say I add the following line in the Makefile:

obj-$(CONFIG_FB_VIA) += via/

and the following line in the Kconfig:

config FB_VIA
tristate “VIA UniChrome/Chrome 9 HC display support”
depends on FB && PCI
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select FRAMEBUFFER_CONSOLE
help
This is the frame buffer device driver for the VIA CLE266, CN400, CN700, CN800,
CN896, CX700, PMN800, PMN880, P4M800CE-Pro, P4M890, P4M900, VN800, VN896 and
VX700 chipsets.

The above are examples from the www.viaarena.com website.

That’s it! Our kernel is now ready to be compiled and installed to get the framebuffer support for the VIA graphic chipset. However, what if there are several users who want to do the same since they also happen to have the same VIA graphic chipset and the driver has still not made into the mainstream kernel.

Here are the steps to generate a patch after your making your own Kernel modifications:

Step 1: Download and prepare kernel sources

Download again the 2.6.24 kernel source (same as you did above) against which you want to build the patch and name directory to something like linux-pristine. For example, your /usr/src directory should now look something like this:

debian:/usr/src# ls -l

total 57316

drwxr-xr-x 20 root root 4096 2008-02-11 03:48 linux-pristine

drwxr-xr-x 20 root root 4096 2008-02-11 03:48 linux-source-2.6.24

lrwxrwxrwx 1 root src 19 2008-04-06 00:20 linux -> linux-source-2.6.24

debian:/usr/src#

Basically, you now have two kernel source 2.6.24 directories:

One with changes that you made to incorporate VIA’s framebuffer driver called as linux (a link to linux-source-2.6.24 directory)

and

the other original 2.6.24 kernel sources (straight from www.kernel.org website) that you just downloaded called as linux-pristine

Step 2: Clean both the kernel directories

We need to make sure that we remove any object or executables files in both the kernel directories. Give the following commands:

debian:/usr/src# cd linux
debian:/usr/src/linux# make mrproper
debian:/usr/src/linux# make clean
debian:/usr/src/linux# cd ../linux-pristine/
debian:/usr/src/linux-pristine# make mrproper
debian:/usr/src/linux-pristine# make clean
debian:/usr/src/linux-pristine#

Step 3: Generate patch

Now give the following commands to create the patch:

debian:# cd /usr/src

debian:/usr/src# diff -Nur linux-pristine/drivers/video/ linux/drivers/video/ > patch-2.6.24-viafb

The format of the diff command is:

diff -options oldfile newfile > patch

where -options = -Nur

oldfile = linux-pristine/drivers/video/

newfile = linux/drivers/video/

patch = patch-2.6.24-viafb

Your patch is now ready and you can view your patch by using your favourite editor like gedit, less, nano, etc.

debian:/usr/src# less patch-2.6.24-viafb

Step 4: Test your patch

We need to make sure that the patch we just generated is correct or now. There is a simple way to do it.

Make a temporary copy of your linux-pristine directory as follow:

debian:/usr/src# cp -r linux-pristine/ /tmp/linux-test-patch

Now patch this kernel source by the patch that you just created in Step 3 above.

debian: cd /tmp/linux-test-patch/

debian:/tmp/linux-test-patch# patch -p1 < /usr/src/patch-2.6.24-viafb

You should be seeing some output as follow:

*******************************
patching file drivers/video/Kconfig
patching file drivers/video/Makefile
patching file drivers/video/via/accel.c
… … … … … … … … … … … … … … … …
patching file drivers/video/via/via_utility.h
patching file drivers/video/via/vt1622a.c
patching file drivers/video/via/vt1622.c
patching file drivers/video/via/vt1625.c
patching file drivers/video/via/vt1636.c
patching file drivers/video/via/vt1636.h
debian:/tmp/linux-test-patch#

*******************************

Finally give the following command to test your patch:

debian:/tmp# diff -rq linux-test-patch/drivers/video/ /usr/src/linux/drivers/video/

If your patch was successfully created in Step 3 than you should not see any output after giving the above command. Basically the above command makes sure that there is no difference in any of the files after you patched your kernel.

Note: If you want to see the explanations of the options of the “diff” command, please see the man page:

debian:/ man diff

Step 5: Tell your users how to use your patch

Just ask your users to patch their 2.6.24 kernel by giving the following command:

debian:/usr/src/linux-source-2.6.24# patch -p1 < /usr/src/patch-2.6.24-viafb

Congratulations! You have successfully created a patch that you can distribute to other users who do not want to make changes manually to the kernel sources as you did initially. Treat yourself with something you like to eat or just sit back and relax and enjoy seeing people using your patch.

As usual, please leave a comment/feedback, if you have any.