Linux Porting Guide

By Rajesh Palani
Embedded Systems Programming
(02/26/01, 07:50:40 PM EDT)

Why pay someone to port Linux for you? Here's oneLinux how-to no embedded programmer should miss.

Linux is gaining in popularity in embedded systems. Many commercial vendors specialize in porting Linux to embedded systems. This article explains the work involved in porting Linux to a specific embedded system and how it was done for one embedded system in particular.

Until now, most embedded operating systems have been proprietary. If a new processor was designed and developed by a semiconductor company, they had to depend on an operating system company to port their operating system to the new processor. The other issue was with the development tools (compiler, debugger, simulator, and so on) for the given processor. Usually the operating system company also provided these tools. In addition, the peripherals around the processor required drivers that had to be developed for the specific operating system. With the introduction of Linux into the embedded sphere, it has become possible for the semiconductor company itself to port an operating system to a new processor, since the source code for the Linux kernel is available as open source. The GNU [1] project provides a wealth of development tools that support Linux and are also open source. In addition, many device drivers are available for Linux, which could be used directly or as a starting point for your target devices.

Project specifics

In our example, the target platform (development board) to which we ported Linux consisted of an ASIC (application-specific integrated circuit) for Windows-based terminals (thin client) and Internet access terminals, a MIPS CPU, and a CPU interface (North Bridge). In addition, the development board also supported EDO DRAM, flash ROM, I2C RTC, I2C EEPROM, and I2C serial bus. The high-level system block diagram is shown in Figure 1. An architectural block diagram of the ASIC is shown in Figure 2. A ramdisk (explained later in the article) served as the root file system. A ramdisk was used initially in order to speed up the porting process. Though this article is based on Linux on MIPS, the overall approach is not very different for other processors (ARM, 386, and so on). In addition, the article deals only with minimum kernel functionality.

Cross-development tools were used for this project. The current development was done on a PC running Red Hat Linux. The Linux VR Web site (see References) is a good starting point for cross-development tools and sources for Linux on MIPS. The following cross-development tools need to be downloaded and installed on the PC running Linux: cross-binutils-as, ld, and so on; C cross compiler; and cross-development C libraries. The detailed steps for installation are available along with the tools.

The kernel sources

The Linux kernel sources for MIPS can be downloaded from the Linux-VR site. One of the most important steps in porting Linux to a new target platform is to have a very clear understanding of how the kernel sources are organized. The following directory structure is not complete and includes only parts that are of interest to this article. The $(TOPDIR) [2] has the following sub-directories:

• arch-This subdirectory contains all of the architecture-specific code. For each supported architecture (MIPS, ARM, 386, and so on), there is a subdirectory under "arch". Each supported architecture subdirectory has four major subdirectories:
• kernel, which contains the architecture-specific kernel code
• mm, which contains the architecture-specific memory management code
• lib, which contains architecture specific library code (vsprintf and so on)
• MY_PLATFORM (target platform directory), which contains platform-specific code.

Note that Linux ports to processors without memory management units (MMU) are also available

• documentation-This subdirectory contains the documentation for the kernel
• drivers-This subdirectory contains code for the device drivers. Each type of device has further subdirectories, such as char, block, net, and so on
• fs-This directory contains the file system code. This has further sub-directories for each supported file system (ext2, proc, and so on)
• include-The include subdirectory contains the include files for the kernel. It has further subdirectories for common include files (for all architectures), one for every architecture supported, and a couple of other subdirectories
• init-This directory contains the initialization code for the kernel
• kernel-This directory contains the main kernel code
• lib-This directory contains the library code of the kernel
• mm-This directory contains the memory management code

Building the kernel image

The Makefile in the $(TOPDIR) has to have ARCH (architecture) properly defined (MIPS in this case). The Makefile in $(TOPDIR)/arch/MY_ ARCH/boot has to have CROSS_COMPILE (mipsel-linux, MIPS little-endian cross-compiler tool-set in this case) and LOADADDR (address at which the kernel image would be loaded) defined as per the configuration. If additional configuration options have to be added, the $(TOPDIR)/arch/MY_ARCH/config.in file has to be modified. You would need to have a config option for your platform (CONFIG_MYPLATFORM) to include code that is specific to your platform. The kernel has to be configured ("make config") to the barest minimum needs (serial driver, ramdisk driver, ext2 file system). Then do a "make dep" to set up the dependencies and finally a "make vmlinux" to produce the kernel image.

Ramdisk

To begin with, a ramdisk can be mounted as the root file system. Ramdisk images and objects are also available readily for MIPS (Linux-VR site). A ramdisk image is a file that contains an image of an ext2 filesystem, while a ramdisk object is an elf object that encapsulates a ramdisk image and can be linked into the kernel. The ramdisk image is usually stored in compressed form. CONFIG_BLK_DEV_RAM and CONFIG_BLK_DEV_INITRD need to be defined as Y in "make config." The ramdisk image can be modified to include your applications, if required. Tools (scripts) are available at the Linux VR site for creating a ramdisk object. The ramdisk.o file needs to be copied to $(TOPDIR)/ arch/MY_ARCH/boot and linked into the kernel. There is a detailed document, $(TOPDIR)/Documenta-tion/ramdisk.txt, on how to use the RAM disk block device with Linux.

Processor-specific changes to the kernel code

If your processor is a standard (or popular) one, in most cases a Linux port to that processor would be available. If you are one of the unlucky few who has to deal with a specific implementation of a given processor core to which Linux has not yet been ported, you'll want to figure out which processor in the list of ported ones is closest to yours and use that port as a starting point for your specific processor implementation. For example, the MIPS core is licensed to many silicon vendors who have their own implementations. The number of TLB [3] (translation lookaside buffers) entries may be different for different implementations. Add a config option (CONFIG_MYCPU) to include code that is specific to your processor. Directories $(TOPDIR)/arch/MY_ARCH/kernel and $(TOPDIR)/arch/MY_ARCH/mm contain the processor-specific code that require modifications if you are dealing with a new implementation.

Assembly file $(TOPDIR)/arch/MY_ARCH/kernel/head.S contains kernel_entry, the entry point for the kernel. This file also contains the exception handling code. Listing 1 shows the implementation of the kernel_entry routine in pseudocode.

1.  Set desired endian mode
2.  Clear the BEV bit
3.  Set the TLB bit
4.  GOTO cpu_probe and return
5.  Set up stack for kernel
6.  Clear bss
7.  GOTO prom_init and return
8.  GOTO loadmmu and return
9.  Disable coprocessors
10. GOTO start_kernel
11. GOTO 10

The configuration register has to be set up correctly. The first thing to be done is to make sure that we are running in the desired endian mode. In our case, we run the system in little-endian mode. The bootstrap exception vector bit needs to be cleared to make sure that the normal exception vectors are used henceforth. In addition, the TLB bit is set to make sure that TLB-based translation is used.

The next step is to probe for the cputype. Listing 2 is a very simple implementation of this function. $(TOPDIR)/include/asm/bootinfo.h contains entries for the cputype (MYCPU) and machine group (MY_MACH_GROUP). The mips_ cputype variable has to be updated in the cpu_probe function. This value is used later to determine the exception handling and MMU routines that need to be loaded for the given CPU, as well as to get the CPU information in the /proc file system.

LEAF(cpu_probe)
la  t3, mips_cputype
li  t2, MYCPU  /* include/asm-mips/bootinfo.h */
b  probe_done
sw  t2, (t3)
END(cpu_probe)

The initial stack for the kernel is set up next. Then the bss section of the kernel image is cleared. Control then transfers to the prom_init() function. Then the TLB and caches are flushed and the cache manipulation functions are set up inside loadmmu(). Disabling of the coprocessors other than coprocessor 0 is done next, followed by a jump to start_kernel().

$(TOPDIR)/arch/MY_ARCH/mm contains the TLB routines and cache handling routines.

Platform specific changes to the kernel code

$(TOPDIR)/arch/MY_ARCH has a sub-directory for each target development platform that is supported. Create a MY_PLATFORM directory by copying a platform closest to your configuration. This directory should contain the interrupt handling, timer, initialization, and setup routines for your specific platform. Create a MY_PLATFORM directory under $(TOPDIR)/include/asm. This directory is used to hold include files specific to your platform.

The prom_init() function, which is part of $(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/prom.c (Listing 3), modifies the command line string to add parameters that need to be passed to the kernel from the bootloader. The machine group and upper bound of usable memory are set up here.

int __init prom_init (int  argc, char **argv, char **envp)
{
unsigned int mem_limit;

// set upper limit to maximum physical RAM (32MB)
mem_limit = 32 * 1024 * 1024;
// the bootloader usually passes us argc/argv[ ]. 
//In the present case, these arguments are not 
//passed from the bootloader. The kernel wants 
// one big string. put it in arcs_cmdline, which 
later gets copied to command_line 
//(see arch/mips/kernel/setup.c)
strcpy (arcs_cmdline, *root=/dev/ram*);

mips_machgroup = MY_MACH_GROUP;
// set the upper bound of usable memory
mips_memory_upper = KSEG0 + mem_limit;
printk(*Detected %dMB of memory\n*, mem_limit >> 20);
return 0;
}

Starting the kernel

Listing 4 contains the first few interesting lines of the start_kernel() function, located in $(TOPDIR)/init/main.c.

asmlinkage void __init start_kernel(void)
{
char * command_line;
/*
* Interrupts are still disabled. Do necessary setups, then
* enable them
*/
lock_kernel();
printk(linux_banner);
setup_arch(&command_line, &memory_start, &memory_end);
memory_start = paging_init(memory_start, memory_end);
trap_init();
init_IRQ();
sched_init();
time_init();
parse_options(command_line);
.
.
.

Listing 5 shows the setup_arch() function in $(TOPDIR)/arch/MY_ARCH/kernel/setup.c. The board-specific setup function is called from here. The command line string and memory start and memory end are passed over to the caller of this function. The start and end addresses for the linked-in ramdisk image are also updated here.

__initfunc(void setup_arch(char **cmdline_p,
unsigned long * memory_start_p, 
unsigned long * memory_end_p))
{
	
#ifdef CONFIG_BLK_DEV_INITRD
#if CONFIG_BLK_DEV_INITRD_OFILE
extern void *__rd_start, *__rd_end;
#endif
#endif
myplatform_setup();
		
strncpy(command_line, arcs_cmdline, CL_SIZE);
*cmdline_p = command_line;
*memory_start_p = (unsigned_long) &_end;
*memory_end_p = mips_memory_upper;
#ifdef CONFIG_BLK_DEV_INITRD
#if CONFIG_BLK_DEV_INITRD_OFILE
// Use the linked-in ramdisk
// image located at __rd_start.
initrd_start = (unsigned long)&__rd_start;
initrd_end = (unsigned long)&__rd_end;
initrd_below_start_ok = 1;
if (initrd_end > memory_end) 
{
printk(*initrd extends beyond end of memory *
*(0x%08lx > 0x%08lx)\ndisabling initrd\n*,
initrd_end, memory_end);
initrd_start = 0;
}
#endif
#endif
}

$(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/setup.c contains the platform-specific initialization code (Listing 6). Here, the various base addresses and the platform-specific RTC and PCI operations are set up. For PCI, the following seven functions need to be implemented for the given platform:

__initfunc(void myplatform_setup(void))
{
irq_setup = myplatform_irq_setup;
	
/*
* mips_io_port_base is the beginning 
*of the address space to which x86  
* style I/O ports are mapped.
*/
mips_io_port_base = 0xa0000000;
/*
* platform_io_mem_base is the 
*beginning of I/O bus memory space as 
* seen from the physical address bus.  
*This may or may not be ident-
* ical to mips_io_port_base, e.g. the 
*former could point to the beginning of PCI 
*memory space while the latter might 
*indicate PCI I/O  space.  The two values are 
*used in different sets of macros.  This 
* must be set to a correct value by the 
*platform setup code.
*/
platform_io_mem_base=0x10000000;
/*
* platform_mem_iobus_base is the 
*beginning of main memory as seen
* from the I/O bus, and must be set 
*by the platform setup code.
*/
platform_mem_iobus_base=0x0;	
	
#ifdef CONFIG_REMOTE_DEBUG
/*
* Do the minimum necessary to set up debugging
*/
myplatform_kgdb_hook(0);
remote_debug = 1;
#endif
#ifdef CONFIG_BLK_DEV_IDE
	ide_ops = &std_ide_ops;
#endif
#ifdef CONFIG_VT
#if defined(CONFIG_DUMMY_CONSOLE)
	conswitchp = &dummy_con;
#endif
#endif
/*
* just set rtc_ops && pci_ops; forget the rest
*/
rtc_ops = &myplatform_rtc_ops;
pci_ops = &myplatform_pci_ops;
}

• myplatform_pcibios_fixup()
• myplatform_pcibios_read_config_byte()
• myplatform_pcibios_read_config_word()
• myplatform_pcibios_read_config_dword()
• myplatform_pcibios_write_config_byte()
• myplatform_pcibios_write_config_word()
• myplatform_pcibios_write_config_dword()

Interrupt handling

The trap_init() function copies the top-level exception handlers to the KSEG0 vector location based on the CPU type. The interrupt handling code is contained in $(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/irq.c and int-handler.S. Most systems use a dedicated interrupt controller to handle the interrupts in the system. The interrupt controller is hooked to one of the external interrupt lines in the processor. The architecture-dependent code has to be modified to fit the interrupt controller into the kernel interrupt handling.

Listing 7 shows the platform-specific interrupt initialization code. The topmost interrupt handler has to be installed using set_except_vector(). The interrupt controller that is used in the platform has to be initialized next. If remote debugging is enabled, a call to set_debug_traps() has to be made to allow any breakpoints or error conditions to be properly intercepted and reported to the debugger. In addition, a breakpoint needs to be generated to begin communication with the debugger running on the host.

static void __init myplatform_irq_setup (void)
{
set_except_vector (0, myplatform_handle_int);
// Initialize InterruptController
InterruptController_Init(IsrTable);
#ifdef CONFIG_REMOTE_DEBUG
printk (*Setting debug traps - 
please connect the remote debugger.\n*);
set_debug_traps ();
breakpoint ();
#endif
}

The top-level interrupt handler (Listing 8) first saves all the registers and then disables further interrupts. The CAUSE register is examined to find the source of the interrupt. If it is a timer interrupt, the corresponding ISR is called. In case it is not a timer interrupt, it checks whether an interrupt has occurred on the line connected to the interrupt controller. The interrupt handler for the interrupt controller (Listing 9) has to get the pending interrupt vector that caused the interrupt and then execute the handler for the particular interrupt source.

NESTED(myplatform_handle_int, PT_SIZE, ra)
	.set  noat
	SAVE_ALL
	CLI
	.set  at
	mfc0  s0, CP0_CAUSE  # get irq mask
	/* First, we check for counter/timer IRQ. */
	andi  a0, s0, CAUSEF_IP5
	beq  a0, zero, 1f
	andi  a0, s0, CAUSEF_IP2  
	# delay slot, check hw0 interrupt
	/* Wheee, a timer interrupt. */
	move  a0, sp
	jal  timer_interrupt
	nop  # delay slot
	j  ret_from_irq
	nop  # delay slot
1:
	beq  a0, zero, 1f
	nop
	/* Wheee, combined hardware 
	level zero interrupt. */
	jal  InterruptController_InterruptHandler
	move  a0, sp  # delay slot
	j  ret_from_irq
	nop  # delay slot
1:
	/* Here by mistake?  This is possible, 
	*what can happen is that by the time we
	*take the  exception the IRQ pin goes low, so 
	 *just leave if this is the case.
	 */
	j  ret_from_irq
	nop
	END(myplatform_handle_int)

void
InterruptController_InterruptHandler (
struct pt_regs *regs
)
{
IntVector intvector;
struct irqaction *action;
int irq, cpu = smp_processor_id();

InterruptControllerGetPendingIntVector(&intvector);
InterruptControllerGetPendingIntSrc((&irq);

action = (struct irqaction *)intvector;
if ( action == NULL ) {
	  printk(*No handler for hw0 irq: %i\n*, irq);
return;
}

hardirq_enter(cpu); 

action->handler(irq, action->dev_id, regs);
kstat.irqs[0][irq]++;
hardirq_exit(cpu);  
} // InterruptController_InterruptHandler ()

The functions request_irq(), free_irq(), enable_irq() and disable_irq() have to be implemented for your target platform. request_irq() is used to install an interrupt handler for a given interrupt source. free_irq() needs to free the memory allocated for the given interrupt. enable_irq() needs to make a call to the interrupt controller function that enables the given interrupt line and disable_irq() needs to disable the given interrupt line.

Timer interrupt

File $(TOPDIR)/arch/MY_ARCH/MY_PLATFORM/time.c contains the platform-dependent timer code. The Linux kernel on MIPS requires a 100Hz timer interrupt. In the MIPS, one of the timers in coprocessor 0 is programmed to generate 100Hz interrupts. The count register and the compare register together make up the timer. When active, the count register contains a free running counter. On each processor clock-tick, the value in the register increments by one. The register is reset and an external hardware interrupt is generated when the values in the count register and compare register match. After the count register is reset, it restarts to count on the next processor clock-tick. The timer interrupt service routine (ISR) needs to call the do_timer() routine. Performing a write to the compare register clears the timer interrupt.

Serial console driver

The console runs on top of a serial driver. A polled serial driver can be used for printk() (kernel debug message) functionality. The minimum functions that this driver needs to provide are the following:

• serial_console_init()-for registering the console printing procedure for kernel printk() functionality, before the console driver is properly initialized
• serial_console_setup()-for initializing the serial port
• serial_console_write(struct console *console, const char *string, int count)-for writing "count" characters

Hook up the serial port on your development board to your host development platform, then start up a serial communication program on your host development platform to communicate with your target.

tty driver

An interrupt driven serial driver can be used to create a terminal device. A terminal device can be created by registering the serial driver with tty. A variety of serial drivers are available in the $(TOPDIR)/drivers/char directory. The driver that matches closest to the serial port hardware being used should be picked up and modified. The interfaces to an interrupt-driven character driver under Linux have been explained in Linux Device Drivers by Rubini (see References).

CONFIG_SERIAL (serial support) has to be defined as Y in "make config." To test, hook up the interrupt-driven serial port to the host development platform and run a serial communication program to communicate with your target (terminal device).

Bootloader

Although LILO (the Linux loader) should be available for your architecture, it may be quicker to use your own bootloader to load the Linux kernel. [4] LILO passes some information to the kernel in a way similar to how an Intel PC BIOS passes information to the kernel. LILO then calls the "kernel_entry" function inside the kernel, giving up control to the kernel. If you're using your own bootloader, you need to pass parameters to the kernel by adding them to the "command_line" string, which is parsed by the kernel. In my case, I had to add "root=/dev/ram" to the command_line string to tell the kernel that I wanted the ramdisk to be mounted as the root file system. You could add other kernel parameters to this string, if needed. Load the image at the specified load address using your bootloader. Start executing from the address of the "kernel_entry" symbol in the kernel image.

It will be easier to debug if the bootloader had its own "print" function, because the printk function inside the kernel buffers all the output to the console until the console is initialized (console_init() in $(TOPDIR)/init/main.c).

If everything goes well, you should get something like the following message on your kernel debug terminal:

Detected 32MB of memory.
Loading R4000/MIPS32 MMU routines.
CPU revision is: 000028a0
Primary instruction cache 32 kb, linesize 32 bytes
Primary data cache 32 kb, linesize 32 bytes
Linux version 2.2.12 (rpalani@rplinux) 
(gcc version egcs-2.90.29 980515 (egcs-10)) 
CPU frequency 200.00 MHz
Calibrating delay loop: 199.88 BogoMIPS
Memory: 14612k/16380k available 
(472k kernel code, 908k data)
Checking for åwait' instruction...  available.
POSIX conformance testing by UNIFIX
Linux NET4.0 for Linux 2.2
Based upon Swansea University Computer Society NET3.039
Starting kswapd v1.1.1.1
No keyboard driver installed
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
RAMDISK: Compressed image found at block 0
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 32k freed

The kernel tries to open a console and find and execute "init" from one of the following places in the root file system, in sequence: /sbin/init, /etc/init, /bin/init. If all the above fail, it tries to create an interactive shell (/bin/sh as happens in my case). If even this fails, then the kernel "panics," as would you. I hope that this does not happen in your case. If it doesn't a shell prompt will appear on the console. Applications can be run on the system by dropping them inside the ramdisk image and executing from there.

Adding new drivers

New drivers for your target hardware can be added by picking up the driver that matches most closely to your hardware (a vast number are available) and modifying it. If you are dealing with a proprietary piece of hardware that is specific to your system, use the standard driver interfaces to implement a driver for the same. These drivers can be implemented as kernel modules in order to load and unload them using insmod and rmmod.

Useful tips

Sprinkle printk() statements liberally throughout your code to aid debugging. This may be an obvious suggestion, but it is worth mentioning. Remote GDB [5] may also be useful for debugging, though in my experience printk's are more than enough for debugging kernel code. In remote GDB, the host development system runs gdb and talks to the kernel running on the target platform via a serial line. You need to setup CONFIG_REMOTE_DEBUG = Y in the kernel configuration. putDebugChar(char ch) and getDebugChar() are the two functions that need to be implemented over the serial port for remote debugging using gdb.

If you are forced to use a common port for console and debug, the GDB output can be multiplexed with the debug output by setting the high bit in putDebugChar(). GDB forwards output without the high bit set to the user session.

To start with, implement only the basic minimum functions for the tty driver as specified in $(TOPDIR)/include/linux/tty_ldisc.h.

Real-time requirements

The subject of embedded systems is not complete without a mention of real-time requirements. The standard Linux kernel provides soft real-time support. There are currently two major approaches to achieve hard real-time with Linux. These are RTLinux and RTAI. Both approaches have their own real-time kernel running Linux as the lowest priority task. When dealing with proprietary hardware, as it often happens in embedded systems, the issue of proprietary software crops up as well. In Linux, proprietary modules can be handled with the GNU Lesser General Public License, which permits linking with non-free modules. It is compatible with the GNU General Public License, which is a free software license, and a copyleft license. [6]

With a good knowledge of the processor architecture and the hardware devices being used, porting Linux to an embedded system can be accomplished in a short time frame, which is of vital importance in the fast paced embedded systems market. In my case, where I have been using UNIX for quite some time, it took me around two months to complete the port of the minimum kernel functionality to our platform. Porting Linux to a different platform should not take that long when doing it for a second time.

Rajesh Palani works as a senior software engineer at Philips Semiconductors. He has been designing and developing embedded software since 1993. He has worked on the design and development of software (ranging from firmware to applications) for set-top boxes, digital still cameras, TVs (Teletext), and antilock braking systems. Contact him at rajesh.palani@philips.com .

Endnotes

1. Stands for "GNU's Not Unix," a project launched in 1984 to develop a complete Unix-like operating system which is free software: the GNU system.
Back

2. The topmost directory in the Linux source tree (/usr/src/linux, by default).
Back

3. Translation Lookaside Buffer-hardware used for virtual to physical address mapping in a processor.
Back

4. The subject of developing a bootloader for your processor is outside the scope of this article.
Back

5. GNU Debugger-helps you to start your program, make it stop on specified conditions, examine what has happened (when your program has stopped), and change things in your program.
Back

6. Copyleft says that anyone who redistributes the software, with or without changes, must pass along the freedom to further copy and change it.
Back

References

A Web site containing a wealth of information on Linux in general:

• www.kernel.org/LDP

Web sites devoted to Linux on MIPS:

• www.paralogos.com/mipslinux
• www.linux-vr.org
• www.linux.sgi.com

Web sites dealing with real-time Linux:

• www.rtlinux.org
• www.rtai.org

Beck, M. et al. Linux Kernel Internals. New York: Addison-Wesley, 1998. This book is a good source of information on the kernel internals. Rubini, Alesandro. Linux Device Drivers. Sebastopol, CA: O Reilly & Associates, 1998. This book delves into kernel internals and talks in detail about all types of device drivers under Linux.