By Rajesh Palani
Embedded Systems Programming
(02/26/01, 07:50:40 PM EDT)
Why pay someone to port Linux for you? Here's oneLinux how-to no embedded programmer should miss.
Linux is gaining in popularity in embedded systems. Many commercial vendors specialize in porting Linux to embedded systems. This article explains the work involved in porting Linux to a specific embedded system and how it was done for one embedded system in particular.
Until now, most embedded operating systems have been proprietary. If a new processor was designed and developed by a semiconductor company, they had to depend on an operating system company to port their operating system to the new processor. The other issue was with the development tools (compiler, debugger, simulator, and so on) for the given processor. Usually the operating system company also provided these tools. In addition, the peripherals around the processor required drivers that had to be developed for the specific operating system. With the introduction of Linux into the embedded sphere, it has become possible for the semiconductor company itself to port an operating system to a new processor, since the source code for the Linux kernel is available as open source. The GNU  project provides a wealth of development tools that support Linux and are also open source. In addition, many device drivers are available for Linux, which could be used directly or as a starting point for your target devices.
In our example, the target platform (development board) to which we ported Linux consisted of an ASIC (application-specific integrated circuit) for Windows-based terminals (thin client) and Internet access terminals, a MIPS CPU, and a CPU interface (North Bridge). In addition, the development board also supported EDO DRAM, flash ROM, I2C RTC, I2C EEPROM, and I2C serial bus. The high-level system block diagram is shown in Figure 1. An architectural block diagram of the ASIC is shown in Figure 2. A ramdisk (explained later in the article) served as the root file system. A ramdisk was used initially in order to speed up the porting process. Though this article is based on Linux on MIPS, the overall approach is not very different for other processors (ARM, 386, and so on). In addition, the article deals only with minimum kernel functionality.
Cross-development tools were used for this project. The current development was done on a PC running Red Hat Linux. The Linux VR Web site (see References) is a good starting point for cross-development tools and sources for Linux on MIPS. The following cross-development tools need to be downloaded and installed on the PC running Linux: cross-binutils-as, ld, and so on; C cross compiler; and cross-development C libraries. The detailed steps for installation are available along with the tools.
The kernel sources
The Linux kernel sources for MIPS can be downloaded from the Linux-VR site. One of the most important steps in porting Linux to a new target platform is to have a very clear understanding of how the kernel sources are organized. The following directory structure is not complete and includes only parts that are of interest to this article. The $(TOPDIR)  has the following sub-directories:
Note that Linux ports to processors without memory management units (MMU) are also available
Building the kernel image
The Makefile in the $(TOPDIR) has to have ARCH (architecture) properly defined (MIPS in this case). The Makefile in $(TOPDIR)/arch/MY_ ARCH/boot has to have CROSS_COMPILE (mipsel-linux, MIPS little-endian cross-compiler tool-set in this case) and LOADADDR (address at which the kernel image would be loaded) defined as per the configuration. If additional configuration options have to be added, the $(TOPDIR)/arch/MY_ARCH/config.in file has to be modified. You would need to have a config option for your platform (CONFIG_MYPLATFORM) to include code that is specific to your platform. The kernel has to be configured ("make config") to the barest minimum needs (serial driver, ramdisk driver, ext2 file system). Then do a "make dep" to set up the dependencies and finally a "make vmlinux" to produce the kernel image.
To begin with, a ramdisk can be mounted as the root file system. Ramdisk images and objects are also available readily for MIPS (Linux-VR site). A ramdisk image is a file that contains an image of an ext2 filesystem, while a ramdisk object is an elf object that encapsulates a ramdisk image and can be linked into the kernel. The ramdisk image is usually stored in compressed form. CONFIG_BLK_DEV_RAM and CONFIG_BLK_DEV_INITRD need to be defined as Y in "make config." The ramdisk image can be modified to include your applications, if required. Tools (scripts) are available at the Linux VR site for creating a ramdisk object. The ramdisk.o file needs to be copied to $(TOPDIR)/ arch/MY_ARCH/boot and linked into the kernel. There is a detailed document, $(TOPDIR)/Documenta-tion/ramdisk.txt, on how to use the RAM disk block device with Linux.
Processor-specific changes to the kernel code
If your processor is a standard (or popular) one, in most cases a Linux port to that processor would be available. If you are one of the unlucky few who has to deal with a specific implementation of a given processor core to which Linux has not yet been ported, you'll want to figure out which processor in the list of ported ones is closest to yours and use that port as a starting point for your specific processor implementation. For example, the MIPS core is licensed to many silicon vendors who have their own implementations. The number of TLB  (translation lookaside buffers) entries may be different for different implementations. Add a config option (CONFIG_MYCPU) to include code that is specific to your processor. Directories $(TOPDIR)/arch/MY_ARCH/kernel and $(TOPDIR)/arch/MY_ARCH/mm contain the processor-specific code that require modifications if you are dealing with a new implementation.
Assembly file $(TOPDIR)/arch/MY_ARCH/kernel/head.S contains kernel_entry, the entry point for the kernel. This file also contains the exception handling code. Listing 1 shows the implementation of the kernel_entry routine in pseudocode.
Listing 1: Kernel entry pseudocode
The configuration register has to be set up correctly. The first thing to be done is to make sure that we are running in the desired endian mode. In our case, we run the system in little-endian mode. The bootstrap exception vector bit needs to be cleared to make sure that the normal exception vectors are used henceforth. In addition, the TLB bit is set to make sure that TLB-based translation is used.
The next step is to probe for the cputype. Listing 2 is a very simple implementation of this function. $(TOPDIR)/include/asm/bootinfo.h contains entries for the cputype (MYCPU) and machine group (MY_MACH_GROUP). The mips_ cputype variable has to be updated in the cpu_probe function. This value is used later to determine the exception handling and MMU routines that need to be loaded for the given CPU, as well as to get the CPU information in the /proc file system.
Listing 2: Code to probe cputype
The initial stack for the kernel is set up next. Then the bss section of the kernel image is cleared. Control then transfers to the prom_init() function. Then the TLB and caches are flushed and the cache manipulation functions are set up inside loadmmu(). Disabling of the coprocessors other than coprocessor 0 is done next, followed by a jump to start_kernel().
$(TOPDIR)/arch/MY_ARCH/mm contains the TLB routines and cache handling routines.
Platform specific changes to the kernel code
$(TOPDIR)/arch/MY_ARCH has a sub-directory for each target development platform that is supported. Create a MY_PLATFORM directory by copying a platform closest to your configuration. This directory should contain the interrupt handling, timer, initialization, and setup routines for your specific platform. Create a MY_PLATFORM directory under $(TOPDIR)/include/asm. This directory is used to hold include files specific to your platform.
The prom_init() function, which is part of $(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/prom.c (Listing 3), modifies the command line string to add parameters that need to be passed to the kernel from the bootloader. The machine group and upper bound of usable memory are set up here.
Listing 3: PROM initialization
Starting the kernel
Listing 4 contains the first few interesting lines of the start_kernel() function, located in $(TOPDIR)/init/main.c.
Listing 4: The beginning of the start_kernel function
Listing 5 shows the setup_arch() function in $(TOPDIR)/arch/MY_ARCH/kernel/setup.c. The board-specific setup function is called from here. The command line string and memory start and memory end are passed over to the caller of this function. The start and end addresses for the linked-in ramdisk image are also updated here.
Listing 5: Architecture setup function
$(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/setup.c contains the platform-specific initialization code (Listing 6). Here, the various base addresses and the platform-specific RTC and PCI operations are set up. For PCI, the following seven functions need to be implemented for the given platform:
Listing 6: Platform-specific initialization code
The trap_init() function copies the top-level exception handlers to the KSEG0 vector location based on the CPU type. The interrupt handling code is contained in $(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/irq.c and int-handler.S. Most systems use a dedicated interrupt controller to handle the interrupts in the system. The interrupt controller is hooked to one of the external interrupt lines in the processor. The architecture-dependent code has to be modified to fit the interrupt controller into the kernel interrupt handling.
Listing 7 shows the platform-specific interrupt initialization code. The topmost interrupt handler has to be installed using set_except_vector(). The interrupt controller that is used in the platform has to be initialized next. If remote debugging is enabled, a call to set_debug_traps() has to be made to allow any breakpoints or error conditions to be properly intercepted and reported to the debugger. In addition, a breakpoint needs to be generated to begin communication with the debugger running on the host.
Listing 7: Platform-specific interrupt initialization
The top-level interrupt handler (Listing 8) first saves all the registers and then disables further interrupts. The CAUSE register is examined to find the source of the interrupt. If it is a timer interrupt, the corresponding ISR is called. In case it is not a timer interrupt, it checks whether an interrupt has occurred on the line connected to the interrupt controller. The interrupt handler for the interrupt controller (Listing 9) has to get the pending interrupt vector that caused the interrupt and then execute the handler for the particular interrupt source.
Listing 8: Top-level interrupt handler
Listing 9: Interrupt handler for the interrupt controller
The functions request_irq(), free_irq(), enable_irq() and disable_irq() have to be implemented for your target platform. request_irq() is used to install an interrupt handler for a given interrupt source. free_irq() needs to free the memory allocated for the given interrupt. enable_irq() needs to make a call to the interrupt controller function that enables the given interrupt line and disable_irq() needs to disable the given interrupt line.
File $(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/time.c contains the platform-dependent timer code. The Linux kernel on MIPS requires a 100Hz timer interrupt. In the MIPS, one of the timers in coprocessor 0 is programmed to generate 100Hz interrupts. The count register and the compare register together make up the timer. When active, the count register contains a free running counter. On each processor clock-tick, the value in the register increments by one. The register is reset and an external hardware interrupt is generated when the values in the count register and compare register match. After the count register is reset, it restarts to count on the next processor clock-tick. The timer interrupt service routine (ISR) needs to call the do_timer() routine. Performing a write to the compare register clears the timer interrupt.
Serial console driver
The console runs on top of a serial driver. A polled serial driver can be used for printk() (kernel debug message) functionality. The minimum functions that this driver needs to provide are the following:
Hook up the serial port on your development board to your host development platform, then start up a serial communication program on your host development platform to communicate with your target.
An interrupt driven serial driver can be used to create a terminal device. A terminal device can be created by registering the serial driver with tty. A variety of serial drivers are available in the $(TOPDIR)/drivers/char directory. The driver that matches closest to the serial port hardware being used should be picked up and modified. The interfaces to an interrupt-driven character driver under Linux have been explained in Linux Device Drivers by Rubini (see References).
CONFIG_SERIAL (serial support) has to be defined as Y in "make config." To test, hook up the interrupt-driven serial port to the host development platform and run a serial communication program to communicate with your target (terminal device).
Although LILO (the Linux loader) should be available for your architecture, it may be quicker to use your own bootloader to load the Linux kernel.  LILO passes some information to the kernel in a way similar to how an Intel PC BIOS passes information to the kernel. LILO then calls the "kernel_entry" function inside the kernel, giving up control to the kernel. If you're using your own bootloader, you need to pass parameters to the kernel by adding them to the "command_line" string, which is parsed by the kernel. In my case, I had to add "root=/dev/ram" to the command_line string to tell the kernel that I wanted the ramdisk to be mounted as the root file system. You could add other kernel parameters to this string, if needed. Load the image at the specified load address using your bootloader. Start executing from the address of the "kernel_entry" symbol in the kernel image.
It will be easier to debug if the bootloader had its own "print" function, because the printk function inside the kernel buffers all the output to the console until the console is initialized (console_init() in $(TOPDIR)/init/main.c).
If everything goes well, you should get something like the following message on your kernel debug terminal:
Detected 32MB of memory.
Loading R4000/MIPS32 MMU routines.
CPU revision is: 000028a0
Primary instruction cache 32 kb, linesize 32 bytes
Primary data cache 32 kb, linesize 32 bytes
Linux version 2.2.12 (rpalani@rplinux)
(gcc version egcs-2.90.29 980515 (egcs-10))
CPU frequency 200.00 MHz
Calibrating delay loop: 199.88 BogoMIPS
Memory: 14612k/16380k available
(472k kernel code, 908k data)
Checking for åwait' instruction... available.
POSIX conformance testing by UNIFIX
Linux NET4.0 for Linux 2.2
Based upon Swansea University Computer Society NET3.039
Starting kswapd v184.108.40.206
No keyboard driver installed
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
RAMDISK: Compressed image found at block 0
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 32k freed
The kernel tries to open a console and find and execute "init" from one of the following places in the root file system, in sequence: /sbin/init, /etc/init, /bin/init. If all the above fail, it tries to create an interactive shell (/bin/sh as happens in my case). If even this fails, then the kernel "panics," as would you. I hope that this does not happen in your case. If it doesn't a shell prompt will appear on the console. Applications can be run on the system by dropping them inside the ramdisk image and executing from there.
Adding new drivers
New drivers for your target hardware can be added by picking up the driver that matches most closely to your hardware (a vast number are available) and modifying it. If you are dealing with a proprietary piece of hardware that is specific to your system, use the standard driver interfaces to implement a driver for the same. These drivers can be implemented as kernel modules in order to load and unload them using insmod and rmmod.
Sprinkle printk() statements liberally throughout your code to aid debugging. This may be an obvious suggestion, but it is worth mentioning. Remote GDB  may also be useful for debugging, though in my experience printk's are more than enough for debugging kernel code. In remote GDB, the host development system runs gdb and talks to the kernel running on the target platform via a serial line. You need to setup CONFIG_REMOTE_DEBUG = Y in the kernel configuration. putDebugChar(char ch) and getDebugChar() are the two functions that need to be implemented over the serial port for remote debugging using gdb.
If you are forced to use a common port for console and debug, the GDB output can be multiplexed with the debug output by setting the high bit in putDebugChar(). GDB forwards output without the high bit set to the user session.
To start with, implement only the basic minimum functions for the tty driver as specified in $(TOPDIR)/include/linux/tty_ldisc.h.
The subject of embedded systems is not complete without a mention of real-time requirements. The standard Linux kernel provides soft real-time support. There are currently two major approaches to achieve hard real-time with Linux. These are RTLinux and RTAI. Both approaches have their own real-time kernel running Linux as the lowest priority task. When dealing with proprietary hardware, as it often happens in embedded systems, the issue of proprietary software crops up as well. In Linux, proprietary modules can be handled with the GNU Lesser General Public License, which permits linking with non-free modules. It is compatible with the GNU General Public License, which is a free software license, and a copyleft license. 
With a good knowledge of the processor architecture and the hardware devices being used, porting Linux to an embedded system can be accomplished in a short time frame, which is of vital importance in the fast paced embedded systems market. In my case, where I have been using UNIX for quite some time, it took me around two months to complete the port of the minimum kernel functionality to our platform. Porting Linux to a different platform should not take that long when doing it for a second time.
Rajesh Palani works as a senior software engineer at Philips Semiconductors. He has been designing and developing embedded software since 1993. He has worked on the design and development of software (ranging from firmware to applications) for set-top boxes, digital still cameras, TVs (Teletext), and antilock braking systems. Contact him at firstname.lastname@example.org .
1. Stands for "GNU's Not Unix,"
a project launched in 1984 to develop a complete Unix-like operating system
which is free software: the GNU system.
2. The topmost directory in
the Linux source tree (/usr/src/linux, by default).
3. Translation Lookaside Buffer-hardware
used for virtual to physical address mapping in a processor.
4. The subject of developing
a bootloader for your processor is outside the scope of this article.
5. GNU Debugger-helps you
to start your program, make it stop on specified conditions, examine what
has happened (when your program has stopped), and change things in your program.
6. Copyleft says that anyone
who redistributes the software, with or without changes, must pass along
the freedom to further copy and change it.
A Web site containing a wealth of information on Linux in general:
Web sites devoted to Linux on MIPS:
Web sites dealing with real-time Linux:
Beck, M. et al. Linux Kernel Internals. New York: Addison-Wesley, 1998. This book is a good source of information on the kernel internals. Rubini, Alesandro. Linux Device Drivers. Sebastopol, CA: O Reilly & Associates, 1998. This book delves into kernel internals and talks in detail about all types of device drivers under Linux.