BOOKS FOR PROFESSIONALS BY PROFESSIONALS ®
Pro Linux Embedded Systems
Dear Reader,
Gene Sally
See last page for details
on $10 eBook version
SOURCE CODE ONLINE
ISBN 978-1-4302-7227-4
5 49 9 9
US $49.99
Shelve in:
Linux
User level:
Intermediate–Advanced
9 781430 272274
Pro
Sally
www.apress.com
RELATED TITLES
Companion eBook
Companion
eBook
Available
Linux Embedded Systems
The tools that you need to succeed with your next embedded Linux project are
hiding in plain sight. So in this book I'll reveal them so you can see for yourself
how to use Linux and open source software to deliver an embedded device.
While you probably have plenty of knowledge about Linux on enterprise servers and desktops, I'll show you how you can in fact use that same set of skills, and
the same tools to you already know, to create embedded projects in Linux. And
that's not just the development tools, such as Eclipse or your favorite editor, but
also the languages you already know. You'll soon discover that embedded development is no longer the province of just C and assembler!
Linux is becoming a common default choice for new embedded commercial projects, with hardware vendors providing board-level support for Linux by
default. That's great news for you as the developer, because this level of "out of
the box" support for Linux on many new devices means that you can often skip
getting Linux running on the board, and go straight to the most interesting parts
of your application development!
This book focuses on the systems and application level tasks that every
embedded engineer must tackle once Linux is up and running in order to deliver
a product. It explains how a system boots so that you can make it faster for your
application, how to build the basic set of software to boot a board, how to configure and build the kernel, and how to be a good citizen when participating in open
source projects. Topics such as minimizing boot time and reducing resource
requirements of Linux receive special treatment.
Lastly, if you're new to embedded development, here's some more good news:
it's just more fun than other types of software engineering. Enjoy!
THE EXPERT’S VOICE ® IN LINUX
Pro
Linux Embedded
Systems
Your complete guide to developing
embedded Linux systems
Gene Sally
Pro Linux Embedded
Systems
■■■
Gene Sally
Pro Lin u x E m bedd ed S yst em s
Copyright © 2010 by Gene Sally
All rights reserved. No part of this work may be reproduced or transmitted in any form or by
any means, electronic or mechanical, including photocopying, recording, or by any
information storage or retrieval system, without the prior written permission of the copyright
owner and the publisher.
ISBN-13 (pbk): 978-1-4302-7227-4
ISBN-13 (electronic): 978-1-4302-7226-7
Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1
Trademarked names may appear in this book. Rather than use a trademark symbol with every
occurrence of a trademarked name, we use the names only in an editorial fashion and to the
benefit of the trademark owner, with no intention of infringement of the trademark.
President and Publisher: Paul Manning
Lead Editor: Michelle Lowman
Technical Reviewer: William von Hagen
Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Gary
Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie,
Duncan Parkes, Jeffrey Pepper, Frank Pohlmann, Douglas Pundick, Ben Renow-Clarke,
Dominic Shakeshaft, Matt Wade, Tom Welsh
Coordinating Editor: Jim Markham
Copy Editor: Tiffany Taylor
Compositor: Bronkella Publishing LLC
Indexer: nSight Indexers
Artist: April Milne
Cover Designer: Anna Ishchenko
Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street,
6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail
[email protected], or visit http://www.springeronline.com.
For information on translations, please contact Apress directly at 2855 Telegraph Avenue,
Suite 600, Berkeley, CA 94705. Phone 510-549-5930, fax 510-549-5939, e-mail
[email protected],
or visit http://www.apress.com.
Apress and friends of ED books may be purchased in bulk for academic, corporate, or
promotional use. eBook versions and licenses are also available for most titles. For more
information, reference our Special Bulk Sales–eBook Licensing web page at
http://www.apress.com/info/bulksales.
The information in this book is distributed on an “as is” basis, without warranty. Although
every precaution has been taken in the preparation of this work, neither the author(s) nor
Apress shall have any liability to any person or entity with respect to any loss or damage
caused or alleged to be caused directly or indirectly by the information contained in this work.
The source code for this book is available to readers at http://www.apress.com.
ii
■ CONTENTS
For Tammi and our sons, Jonah, Benjamin, and Adam.
iii
■ CONTENTS
Contents at a Glance
Content s at a Glance .......................................................................... iv
Content s ............................................................................................ v
About the Author .............................................................................. xvi
About the Technical R eviewer ........................................................... xvii
Acknowledgment s ........................................................................... xviii
Introduct ion ..................................................................................... xix
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
■Chapter
iv
1: A bout Embedded Linux .................................................... 1
2: C onfiguring the Soft ware Envir onment ............................ 25
3: Target Emulation and Virtual Machines ........................... 55
4: St arting Your Project ..................................................... 63
5: G etting Linux for Your Board .......................................... 83
6: Cr eating a Linux Distr ibution fr om Scrat ch .................... 107
7: B ooting the B oard ....................................................... 143
8: C onfiguring the Application D evelopment Envir onment ... 169
9: A pplication Development ............................................. 197
10: D ebugging A pplications ............................................. 215
11: Kernel Configuration and Development ........................ 237
12: R eal Time ................................................................. 257
13: U sing Open Source Soft ware Project s ......................... 273
14: B usyB ox ................................................................... 293
15: Syst em Design .......................................................... 309
16: Syst em Tuning .......................................................... 335
17: D eploying Applicat ions .............................................. 363
18: H andling Field U pdat es .............................................. 383
■ CONTENTS
Contents
Content s at a Glance .......................................................................... iv
Content s ............................................................................................ v
About the Author .............................................................................. xvi
About the Technical R eviewer ........................................................... xvii
Acknowledgment s ........................................................................... xviii
Introduct ion ..................................................................................... xix
■Chapter 1: A bout Embedded Linux ..................................................... 1
Why Use Embedded Linux?............................................................................................. 2
Technical Reasons to Use Embedded Linux...........................................................................................2
Commercial Reasons to Use Embedded Linux.......................................................................................7
10,000-Foot Embedded Linux Development Flyover ...................................................... 9
Target Hardware ....................................................................................................................................9
Obtaining Linux ....................................................................................................................................10
Booting Linux .......................................................................................................................................10
Development Environment ...................................................................................................................10
System Design .....................................................................................................................................11
Anatomy of an Embedded Linux System....................................................................... 11
Boot Loader..........................................................................................................................................12
Kernel...................................................................................................................................................13
Root File System ..................................................................................................................................13
Your Application ...................................................................................................................................14
Cross-Compiler ....................................................................................................................................14
Tools of the Trade ................................................................................................................................15
v
■ CONTENTS
Where to Get Help ......................................................................................................... 19
University of Google .............................................................................................................................19
Mailing Lists and Newsgroups .............................................................................................................19
Vendor-Sponsored Resources..............................................................................................................20
Trade Group and Community Interest Sites .........................................................................................21
IRC........................................................................................................................................................23
Next Up ......................................................................................................................... 23
■Chapter 2: C onfiguring the Soft ware Envir onment ............................. 25
Host Environment.......................................................................................................... 26
Linux ....................................................................................................................................................26
Windows ..............................................................................................................................................29
Host Services ................................................................................................................ 43
Turn Off Your Firewall ..........................................................................................................................44
TFTP .....................................................................................................................................................44
DHCP ....................................................................................................................................................45
NFS.......................................................................................................................................................47
PXE.......................................................................................................................................................50
Cabling .......................................................................................................................... 51
Serial (for Console)...............................................................................................................................51
Network................................................................................................................................................51
Avoiding an Angry Visit from IT..................................................................................... 52
Dual-Homed Host .................................................................................................................................52
■Chapter 3: Target Emulation and Virtual Machines ........................... 55
Why Target Emulation? ................................................................................................. 55
Emulation via QEMU...................................................................................................... 56
Compiling QEMU...................................................................................................................................56
Using QEMU to Emulate a Target .........................................................................................................58
Using QEMU to Compile under Emulation ............................................................................................60
vi
■ CONTENTS
Virtualization Software for x86 Hosts............................................................................ 61
Approaches to Virtualization ................................................................................................................61
Summary....................................................................................................................... 62
■Chapter 4: St arting Your Project ...................................................... 63
Most Boards Include a Linux Distribution ..................................................................... 64
What to Do After Unpacking the Board ......................................................................... 65
Have Linux? Boot It! .............................................................................................................................66
Assess the Kernel.................................................................................................................................73
Understand the RFS .............................................................................................................................76
Suitability for Your Project ............................................................................................ 80
Cross-Compiler ....................................................................................................................................81
Moving Forward ............................................................................................................ 82
■Chapter 5: G etting Linux for Your Board ........................................... 83
Obtaining Linux from the Board Vendor ........................................................................ 84
Questions You Should Ask Your Board Vendor.....................................................................................84
Now That You’re a Customer… ...........................................................................................................87
Open Source Embedded Distributions .......................................................................... 87
Why Embedded Linux Distribution Builders Exist ................................................................................88
Should You Use One? ...........................................................................................................................88
Popular Open Source Embedded Distributions ....................................................................................89
Getting Linux from Commercial Vendors and Consultants.......................................... 102
Do You Need a Commercial Vendor? .................................................................................................102
What You Should Expect ....................................................................................................................103
Roundup of Vendors...........................................................................................................................104
What’s Next................................................................................................................. 106
■Chapter 6: Cr eating a Linux Distr ibution f rom Scrat ch .................... 107
Cross-Compiler Basics................................................................................................ 108
A Note about Building Software .........................................................................................................109
Get Comfortable with the Command Line ..........................................................................................110
vii
■ CONTENTS
Overview of Building a GCC Cross-Compiler............................................................... 111
The C Library ......................................................................................................................................112
Gathering Sources..............................................................................................................................112
Building GCC ......................................................................................................................................117
Building Toolchains with Crosstool-NG....................................................................... 130
Creating the Root File System..................................................................................... 131
Configuring the Environment .............................................................................................................132
Building and Installing BusyBox.........................................................................................................132
Libraries .............................................................................................................................................133
Creating Device Nodes and Directories..............................................................................................134
Finishing Touches ..............................................................................................................................134
Building the Kernel.............................................................................................................................135
Troubleshooting Booting Problems ....................................................................................................138
Distributing the Distribution........................................................................................ 140
Wrapping Up ............................................................................................................... 141
■Chapter 7: B ooting the B oard ........................................................ 143
Booting a Linux System Is a Three-Act Play ............................................................... 143
The Boot Loader .................................................................................................................................143
Kernel-Land vs. Userland ...................................................................................................................146
Boot Loaders ............................................................................................................... 147
About Flash Memory ..........................................................................................................................152
Kernel Startup............................................................................................................. 153
The Kernel Entry Point........................................................................................................................154
Userland Startup ................................................................................................................................161
BusyBox Init .......................................................................................................................................165
Your Init..............................................................................................................................................165
What’s Next................................................................................................................. 167
viii
■ CONTENTS
■Chapter 8: C onfiguring the Application D evelopment Envir onment .... 169
Pick the Right Tool for the Job.................................................................................... 169
Know Your Application ................................................................................................ 169
Hardware Constraints ........................................................................................................................170
What to Use for Development ..................................................................................... 172
C.........................................................................................................................................................172
C++ ....................................................................................................................................................173
Java....................................................................................................................................................173
Non-Traditional Embedded Languages....................................................................... 175
Python ................................................................................................................................................175
TCL .....................................................................................................................................................177
Shell Scripting....................................................................................................................................178
PHP.....................................................................................................................................................179
Performance and Profiling Tools................................................................................. 180
Profiling..............................................................................................................................................180
Leak Detection ...................................................................................................................................184
Static Analysis....................................................................................................................................187
IDE............................................................................................................................... 188
Your Editor + Make + Shell................................................................................................................188
Eclipse................................................................................................................................................191
What’s Next................................................................................................................. 196
■Chapter 9: A pplication Development .............................................. 197
Coding for Portability .................................................................................................. 198
System Differences..................................................................................................... 199
FIFO ....................................................................................................................................................199
Getting the Tools ......................................................................................................... 201
Making Make Work ..................................................................................................... 201
Running the Code on the Target ................................................................................. 205
More Complex Projects ............................................................................................... 206
ix
■ CONTENTS
■Chapter 10: D ebugging A pplications .............................................. 215
Getting Started on Your Application............................................................................ 215
Types of Debugging .................................................................................................... 215
Remote Debugging Overview...................................................................................... 216
Debugging C and C++................................................................................................. 217
Building GDB ......................................................................................................................................217
GDB Front Ends ..................................................................................................................................218
Compiling for Debugging ...................................................................................................................219
Debugging Java .......................................................................................................... 229
Instrumentation........................................................................................................... 233
Java Instrumentation .................................................................................................. 235
Instrumentation in Scripting Languages ..................................................................... 236
What’s Next................................................................................................................. 236
■Chapter 11: Kernel Configuration and Development ........................ 237
Kernel Project Layout .................................................................................................. 237
Downloading the Kernel.....................................................................................................................239
Building the Kernel...................................................................................................... 241
How Kernel Configuration Works .......................................................................................................244
Default Configurations .......................................................................................................................245
Editing .config By Hand ......................................................................................................................247
Building the Kernel.............................................................................................................................247
Building Modules ...............................................................................................................................250
Cleaning Up ........................................................................................................................................251
Open Source Community ............................................................................................ 252
The Kernel Development Process ......................................................................................................252
Contributing to the Linux Kernel ........................................................................................................252
Applying Patches................................................................................................................................254
What’s Next........................................................................................................................................255
x
■ CONTENTS
■Chapter 12: R eal Time .................................................................. 257
Real-Time Core Concepts...................................................................................................................258
The Linux Scheduler...........................................................................................................................260
Real-Time Scheduler..........................................................................................................................260
Real-Time Implementation in Linux ............................................................................ 261
Getting the Patch................................................................................................................................262
Real-Time Programming Practices ............................................................................. 264
The One Real-Time Process ...............................................................................................................264
Lock Memory .....................................................................................................................................264
Avoid the Heap ...................................................................................................................................265
Asking for Priority Inheritance Mutexes.............................................................................................265
I/O Is Nondeterministic.......................................................................................................................266
Using Thread Pools ............................................................................................................................266
LatencyTOP ........................................................................................................................................267
Common Hardware Pitfalls ......................................................................................... 270
System Management Interrupts.........................................................................................................270
VGA Console .......................................................................................................................................270
DMA Bus Mastering............................................................................................................................270
Summary..................................................................................................................... 271
■Chapter 13: U sing Open Source Soft ware Project s ......................... 273
Using Open Source Packages ..................................................................................... 273
How an Open Source Project Is Structured ........................................................................................274
The Project Team Isn’t Your Extended Workforce..............................................................................275
Understand the Licensing ..................................................................................................................275
Downloading ......................................................................................................................................277
Using Source Control Systems to Fetch Code....................................................................................278
Cross-Compiling.................................................................................................................................282
Using configure ..................................................................................................................................282
Building and Installing........................................................................................................................288
xi
■ CONTENTS
Commonly Used Projects ............................................................................................ 289
DirectFB .............................................................................................................................................289
Dropbear ............................................................................................................................................289
QT/Qtopia ...........................................................................................................................................289
JamVM ...............................................................................................................................................289
Rzsz....................................................................................................................................................290
Netcat.................................................................................................................................................290
TinyXML .............................................................................................................................................290
Micro_httpd........................................................................................................................................290
Stupid-FTPd .......................................................................................................................................291
Quagga...............................................................................................................................................291
Tslib ...................................................................................................................................................291
fgetty..................................................................................................................................................291
■Chapter 14: B usyB ox .................................................................... 293
How a BusyBox-Based System Is Structured ............................................................. 293
Building a BusyBox-Based System ............................................................................. 294
Download the Software......................................................................................................................295
Configure............................................................................................................................................295
What Makes BusyBox Small?...................................................................................... 302
Creating Your Own Applet ........................................................................................... 303
Create the Applet Code ......................................................................................................................303
Getting Help ................................................................................................................ 307
What’s Next................................................................................................................. 307
■Chapter 15: Syst em Design ........................................................... 309
The Big Picture............................................................................................................ 309
Configuring the Boot Loader and Kernel ..................................................................... 310
U-Boot ................................................................................................................................................310
Other Boot loaders .............................................................................................................................313
Execute in Place.................................................................................................................................313
xii
■ CONTENTS
Selecting a Root File System ...................................................................................... 313
Block-Based File Systems .................................................................................................................314
MTD File Systems ..............................................................................................................................318
RAM Buffer–Based File Systems........................................................................................................319
File System Pairings...........................................................................................................................320
Assembling a Root File System................................................................................... 320
Create the Staging Area .....................................................................................................................320
Create a Directory Skeleton ...............................................................................................................321
Gather Libraries and Required Files...................................................................................................321
Create Initialization Scripts ................................................................................................................323
Set Ownership and Permissions ........................................................................................................327
Security....................................................................................................................... 328
Built-In Security .................................................................................................................................328
SELinux ..............................................................................................................................................329
PAM....................................................................................................................................................332
What’s Next................................................................................................................. 334
■Chapter 16: Syst em Tuning ........................................................... 335
Three or Fewer Megabytes ................................................................................................................335
16–32 Megabytes ..............................................................................................................................336
More than a Gigabyte.................................................................................................. 336
Reducing the Size of the Root File System ........................................................................................337
Start from Zero...................................................................................................................................337
Compiling to Save Space ............................................................................................ 340
Reducing the Size of the Kernel.........................................................................................................343
Removing Unneeded Features and Drivers................................................................. 344
Minimizing Boot Time ........................................................................................................................349
Reducing Kernel Boot-Up Time ..........................................................................................................350
Measuring Kernel Boot-Up Times ......................................................................................................352
Reducing Root File System Startup Times.................................................................. 356
What’s Next................................................................................................................. 361
xiii
■ CONTENTS
■Chapter 17: D eploying Applicat ions ............................................... 363
Deployment for Embedded Devices ............................................................................ 363
Requirements.....................................................................................................................................364
Industrial Design ................................................................................................................................365
Mechanical Design.............................................................................................................................365
Electrical Engineering ........................................................................................................................366
Manufacturing Engineering................................................................................................................367
Software Design.................................................................................................................................367
Software Engineering.........................................................................................................................368
Manufacturing....................................................................................................................................369
Deployment Strategies and Tactics ............................................................................ 371
Boot Loaders ............................................................................................................... 372
In General...........................................................................................................................................372
UBOOT: Configuring Initial Parameters ..............................................................................................373
Expect ................................................................................................................................................374
Boot Loaders Are Just Programs .......................................................................................................377
Deployment Root File Systems ................................................................................... 378
Application Files and Libraries...........................................................................................................379
First Field Update at the Factory ........................................................................................................381
What’s Next................................................................................................................. 381
■Chapter 18: H andling Field U pdat es ............................................... 383
Root File System Updates ........................................................................................... 383
Basic Strategies .................................................................................................................................384
Forklift Upgrade .................................................................................................................................384
Parallel Systems ................................................................................................................................388
Do It Yourself......................................................................................................................................389
Using Package Managers...................................................................................................................390
Initramfs Root File Systems ...............................................................................................................401
Kernel Updates............................................................................................................ 401
Basic Strategies .................................................................................................................................402
xiv
■ CONTENTS
Modules .............................................................................................................................................403
Forklift................................................................................................................................................405
Field Update Failures .................................................................................................. 406
Report Failure, Stop ...........................................................................................................................406
Failsafe Root File System...................................................................................................................406
Failsafe Kernel ...................................................................................................................................406
In Summary................................................................................................................. 407
■Index .......................................................................................... 409
xv
■ CONTENTS
About the Author
■ Gen e Sa lly got mixed-up with computers at a young age, his fascination sparked by an Apple II, with
the Lemon cooling system add-on, no less. As a software professional, Gene got his first job writing make
files and then moved on to more exciting (for certain values of exciting) things like accounting,
insurance processing, and social services systems. He first used Linux to set up a shared Internet
connection and later used it when working on software that tested telecommunication management
software; Gene was happy that a decent Unix-like environment could be had for free, and Linux became
his axe of choice. Gene next found himself at a start-up that specialized in tools and distributions for
embedded Linux, working in a variety of roles including engineer, trainer, technical support phoneanswerer-guy, podcaster, and marketer. Presently, Gene is working at a company that creates safety
products for retirement and assisted-living homes.
Gene resides outside of Pittsburgh, Pennsylvania in the rolling hills north of the city with his wife and
three sons. When not working, writing, playing with kids, or tending to the never-ending repair list at
home, Gene cycles, skis, and hikes throughout western Pennsylvania and eastern Ohio.
xvi
■ CONTENTS
About the Technical Reviewer
■ Wil li a m von Hag en (Bi ll) has been a UNIX system administrator for over 20
years and a Linux fanatic since the early 1990s. He has worked as a systems
programmer, system administrator, writer, application developer, drummer, and
documentation manager. Bill has written or pco-written books on such topics as
Ubuntu Linux, GCC, Linux Server Hacks, Linux Filesystems, SUSE Linux, Red Hat
Linux, SGML, and Mac OS X.
xvii
■ CONTENTS
Acknowledgments
First, I thank the good Lord for providing me with time, prosperity, curiosity, vigor, intellect, and health;
I hope this work puts these gifts to proper use. During the time I spent writing this book, I received
tremendous support from my family. I couldn’t have done it without my wife and kids: you helped more
than you know.
Writing a book is a team effort, and I am fortunate that the team working on this book was as excellent as
any author could have ever expected. Thank you Bill von Hagen for being my technical reviewer, and
Michelle Lowman and Frank Pohlmann for being my editors. All of the editors actually read what was
written and provided feedback that made this book much better than it would have been without their
efforts. I would also like to thank the production team at Apress, who created the cover, laid out the
pages, and amazingly turned a collection of documents into the book you’re holding now. Special thanks
goes to James Markham, who somehow managed to tolerate my inability to meet a deadline with more
patience than I deserved. Last, but not least, the open source community has provided me with an
excellent education about Linux, both in mail and newsgroup traffic and in the source code itself.
xviii
.PDF 
445 
473 
131 
