Linux: the big picture

（source link: http://liw.iki.fi/liw/texts/linux-the-big-picture.html）

This article gives a brief introduction to Linux, with a sketch of thebackground history. It was written forPC Update, the monthly magazineof thePC user group in Melbourne,Australia. This version does not contain all the editorial changes andspelling fixes made by the magazine.

April 28, 2003

History

The history of computer operating systems starts in the1950s, with simple schemes for running batch programs efficiently,minimizing idle time between programs. A batch program is onethat does not interact with the user at all. It reads all itsinput from a file (possibly a stack of punch cards) and outputsall its output to another file (possibly to a printer). This ishow all computers used to work.

Then, in early 1960s, interactive use started to gainground. Not only interactive use, but having several people usethe same computer at the same time, from different terminals. Suchsystems were called time-sharing systems and were quite achallenge to implement compared to the batch systems.

During the 1960s there were many attempts at building goodtime-sharing systems. Some of these were university researchprojects, others were commercial ones. One such project wasMultics, which was quite innovative at the time. It had, forexample, a hierarchical file system, something taken for grantedin modern operating systems.

The Multics project did not, however, progress very well. Ittook years longer to complete than anticipated and never got asignificant share of the operating system market. One of theparticipants, Bell Labs, withdrew from the project. The BellLabs people who were involved then made their own operatingsystem and called it Unix.

Unix was originally distributed for free and gained muchpopularity in universities. Later, it got an implementationof the TCP/IP protocol stack and was adopted as the operatingsystem of choice for early workstations.

By 1990, Unix had a strong position in the server marketand was especially strong in universities. Most universities hadUnix systems and computer science students were exposed to them.Many of them wanted to run Unix on their own computers as well.Unfortunately, by that time, Unix had become commercial andrather expensive. About the only cheap option was Minix,a limited Unix-like system written by Andrew Tanenbaum forteaching purposes. There was also 386BSD, a precursor NetBSD,FreeBSD, and OpenBSD, but that wasn't mature yet, and requiredhigher end hardware than many had at home.

Into this scene came Linux, in October, 1991. LinusTorvalds, the author, had used Unix at the University ofHelsinki, and wanted something similar on his PC at home. Sincethe commercial alternatives were way too expensive, he startedout with Minix, but wanted something better and soon startedto write his own operating system. After its first release,it soon attracted the attention of several other hackers. WhileLinux initially was not really useful except as a toy, it soongathered enough features to be interesting even for peopleuninterested in operating system development.

Linux itself is only the kernel of an operatingsystem. The kernel is the part that makes all other programsrun. It implements multitasking, and manages hardware devices,and generally enables applications to do their thing. Allthe programs that the user (or system administrator) actuallyinteracts with are run on top of the kernel. Some of these areessential: for example, a command line interpreter (or shell),which is used both interactively and to write shell scripts(corresponding to .BAT files).

Linus did not write these programs himself, and usedexisting free versions instead. This reduced greatly the amountof work he had to do to get a working environment. In fact, heoften changed the kernel to make it easier to get the existingprograms to run on Linux, instead of the other way around.

Most of the critically important system software, includingthe C compiler, came from the Free Software Foundation's GNUproject. Started in 1984, the GNU project aims to develop anentire Unix-like operating system that is completely free. Tocredit them, many people like to refer to a Linux system as aGNU/Linux system. (GNU has their own kernel as well.)

During 1992 and 1993, the Linux kernel gathered all thenecessary features it required to work as a replacement forUnix workstations, including TCP/IP networking and a graphicalwindowing system (the X Window System). Linux also receivedplenty of industry attention, and several small companies werestarted to develop and distribute Linux. Dozens of user groupswere founded, and the Linux Journal magazine started to appearin early 1994.

Version 1.0 of the Linux kernel was released in March,1994. Since then, the kernel has gone through many developmentcycles, each culminating in a stable version. Each developmentcycle has taken a year or three, and has involved redesigningand rewriting large parts of the kernel to deal with changes inhardware (for example, new ways to connect peripherals, such asUSB) and to meet increased speed requirements as people applyLinux to larger and larger systems (or smaller and smaller ones:embedded Linux is becoming a hot topic).

From a marketing and political point of view, after the1.0 release the next huge step happened in 1997, when Netscapedecided to release their web browser as free software (the term'open source' was created for this). This was the occasionthat first brought free software to the attention of the wholecomputing world for the time. It has taken years of work sincethen, but free software (whether called that or open source) hasbecome not only generally accepted but also often the preferredchoice for many applications.

Social phenomenon

Apart from being a technological feat, Linux is also aninteresting social phenomenon. Much through Linux, the freesoftware movement has broken through to general attention. Onthe way, it even got an informal marketing department and brand:open source. It is baffling to many outsiders that something assuccessful as Linux could be developed by a bunch of unorganizedpeople in their free time.

The major factor here is the availability of all thesource code to the system, plus a copyright license that allowsmodifications to be made and distributed. When the system hasmany programmers among its users, if they find a problem, theycan fairly easily fix it. Additionally, if they think a featureis missing, they can add it themselves. For some reason, thatis something programmers like to do, even if they're not paidfor it: they have an itch (a need), so they scratch (write thecode to fill the need).

It is necessary to have at least one committed developer whoputs in lots of effort. After a while, however, once there areenough programmer-users sending small changes and improvements,you get a snowball effect: lots of small changes result in afairly rapid total development speed, which then attracts moreusers, some of which will be programmers. This then resultsin more small changes and improvements sent in by users, andso on.

For operating system development specifically, this largegroup of programmer-users results in two important types ofimprovements: bug fixes and device drivers. Operating system codeoften has bugs that only occur rarely and it can be difficultfor the developers to reproduce them. When there are thousandsor more users who are also programmers, this results in a veryeffective testing and debugging army.

Most of the code volume in Linux is device drivers. Thecore functionality, which implements multitasking and multiuserfunctionality, is small in comparison. Most device drivers areindependent from each other, and only interact with the operatingsystem core via well defined interfaces. Thus, it is fairlyeasy to write a new device driver without having to understandthe whole complexity of the operating system. This also allowsthe main developers to concentrate on the core functiionality,and they can let those people write the device drivers whoactually have the devices.

It would be awkward just to store the thousands of differentsound cards, Ethernet cards, IDE controllers, motherboards,digital cameras, printers, and so on that Linux supports. TheLinux development model is distributed, and spreads the workaround quite effectively.

The Linux model is not without problems. When a new devicegets on the market, it can take a few months before a Linuxprogrammer is interested enough to write a device driver. Also,some device manufacturers, for whatever reason, do not want torelease programming information for their devices, which canprevent a Linux device driver to be written at all. Luckily,with the growing global interest in Linux such companies becomefewer in numbers.

What it is

Linux is a Unix-like multitasking, multiuser 32 and 64 bitoperating system for a variety of hardware platforms and licensedunder an open source license. This is a somewhat accurate butrather brief description. I'll spend the rest of this articleexpounding on it.

Being Unix-like means emulating the Unix operatingsystem interfaces so that programs written for Unix will workfor Linux merely by re-compiling. It follows that Linux usesmostly the same abstractions as the Unix system. For example,the way processes are created and controlled is the same in Unixand Linux.

There are a number of other operating systems in activeuse: from Microsoft's family of Windows versions, through Apple'sMacOS to OpenVMS. Linux's creator, Linus Torvalds, chose Unixas the model for Linux partly for its aesthetic appeal to systemprogrammers, partly because of all the operating systems he wasfamiliar with, it was the one he knew best.

The Unix heritage also gives Linux the two most importantfeatures: multitasking and multiuser capabilities. Linux, likeUnix, was designed from the start to run multiple processesindependently of each other. Implementing multitaskingwell requires attention at every level of the operatingsystem. It is hard to add multitasking to an operationg systemafterwards. That's why the Windows 95 series and MacOS (beforeMacOS X) did multitasking somewhat poorly: multitasking wasadded to an existing operating system, not designed into a newone. That's also why the Windows NT series, MacOS X, and Linuxdo multitasking so much better.

A good implementation of multitasking requires, among otherthings, proper memory management. The operating system must usethe memory protection support in the processor to protect runningprograms from each other. Otherwise a buggy program (that is,most any program) may corrupt the memory area of another program,or the operating system itself, causing weird behavior or a totalsystem crash, with likely loss of data and unsaved work.

Supporting many concurrent users is easy after multitaskingworks. You label each instance of a running program with aparticular user and prevent the program from tampering withother user's files.

Portable and scalable

Linux was originally written for an Intel 386 processor, and naturallyworks on all successive processors. After about three years ofdevelopment, work began to adapt (or port) Linux to other processorfamilies as well. The first one was the Alpha processor, then developedand sold by the Digital Equipment Corporation. The Alpha was chosenbecause Digital graciously donated a system to Linus. Soon other portingefforts followed. Today, Linux also runs on Sun SPARC and UltraSPARC,Motorola 68000, PowerPC, PowerPC64, ARM, Hitachi SuperH, IBM S/390,MIPS, HP PA-RISC, Intel IA-64, DEC VAX, AMD x86-64 and CRIS processors.(Seehttp://kernel.org for details.)

Most of those processors are not very common onpeople's desks. For example, S/390 is IBM's big mainframearchitecture. Here, mainframe means the kind of computer insideof which you can put your desk, rather than the kind that fitson your desk.

Some of those processors are 32 bit, like the Intel386. Others are 64 bit, such as the Alpha. Supporting suchdifferent processors has been good for Linux. It has requireddesigning the system to use proper modularity and goodabstractions and this has improved code quality.

The large variety of supported processors also shows offLinux's scalability: it works everything from very small systems,such as embedded computers, handheld devices, and mobile phones,to very large systems, such as the IBM mainframes.

Using clustering technology, such as Beowulf(http://www.beowulf.org/), Linux even runs on supercomputers. Forexample, the US Lawrence Livermore National Laboratories bought acluster with 1920 processors, resulting in one of the five fastestsupercomputers in the world with a theoretical peak performance of 9.2teraFLOPS or 9.2 trillion calculations per second.(http://lwn.net/Articles/4759/).

Using Linux

The operating system itself is pretty boring tomost people. Applications are necessary so to get thingsdone. Traditionally, Linux applications have been the kinds ofapplications used with Unix: scientific software, databases,and network services. Also, of course, all the tools programmerswant for their craft.

Much of such software seems rather old-fashioned bytoday's desktop standards. User interfaces are text based,or they might not exist at all. Indeed, most software hasusually been non-interactive and has been of the command line,batch processing variety. Since most users have been experts inthe application domain, this has been good enough.

Thus, Linux first found corporate employment as a fileserver, mail server, web server, or firewall. It was a goodplatform for running a database, with support from all majorcommercial database manufacturers.

In the past few years Linux has also become an interesting option on theuser friendly desktop front. The KDE (http://www.kde.org/) and Gnome(http://www.gnome.org/) projects develop desktopenvironments and applications that are easy to learn (as well aseffective to use). There is now plenty of desktop applications whichpeople with Windows or MacOS experience will have no difficulty using.

There is even a professional grade office software package. OpenOffice(http://www.openoffice.org/), based on Sun's StarOffice, is free,fully featured, and file compatible with Microsoft Office. It includes aword processor, spreadsheet, and presentation program, competing withMicrosoft's Word, Excel, and Powerpoint.

Linux distributions

To install Linux, you have to choose a Linux distribution. Adistribution is the Linux kernel, plus an installation program,plus some set of applications to run on top of it. There arehundreds of Linux distributions, serving different needs.

All distributions use pretty much the same actual software,but they are different in which software they include, whichversions they pick (a stable version known to work well or thelatest version with all the bells and whistles and bugs), howthe software is pre-configured, and how the system is installedand managed. For example, OpenOffice, Mozilla (web browser),KDE and Gnome (desktop environments), and Apache (web server)will all work on all distributions.

Some distributions aim to be general purpose, but most ofthem are task specific: they are meant for running a firewall,a web kiosk, or meant for users within a particular universityor country. Those looking for their first Linux experience canconcentrate on the three biggest general purpose distributions:Red Hat, SuSE, and Debian.

The Red Hat and SuSE distributions are produced by companiesby the same names. They aim at providing an easy installationprocedure, and for a pleasant desktop experience. They are alsogood as servers. Both are sold in boxes, with an installationCD and printed manual. Both can also be downloaded via thenetwork.

The Debian distribution is produced by a volunteerorganization. It's installation is less easy: you have toanswer questions during the installation the other distributionsdeduce automatically. Nothing complicated as such, but requiringunderstanding of and information about hardware most PC usersdon't want to worry about. On the other hand, after installation,Debian can be upgraded to each new release without re-installinganything.

The easiest way to try out Linux is to use a distributionthat works completely off a CD-ROM. This way, you don't haveto install anything. You merely download the CD-ROM imagefrom the net and burn it on a disk, or buy a mass-produced onevia the net. Insert disk in drive, then reboot. Not having toinstall anything on the hard disk means you can easily switchbetween Linux and Windows. Also, since all Linux files are ona read-only CD-ROM, you can't break anthing by mistake whileyou're learning.

Further information

• The Linux Documentation Project. A good placeto start looking for manuals and help.

• Linux Weekly News, Slashdot:Two important news sites about Linux and related things.

• Linux distributions: http://www.debian.org/,http://www.redhat.com/,http://www.suse.com/us/index.html.

• Knoppix:Single bootable CD, for testing Linux without installing.

Author blurb

Lars Wirzenius designs and implements embedded telematic software forOliotalo at work, and develops Debian at home.

Last edited 2009-06-02 08:12