7_Programming

How To Do Research In the MIT AI Lab

Programming

Not every AI thesis involves code, and there are important people in AI who have never written a significant program, but to a first approximation you have to be able to program to do AI. Not only does most AI work involve writing programs, but learning to program gives you crucial intuitions into what is and isn’t computationally feasible, which is the major source of constraint AI contributes to cognitive science.

At MIT, essentially all AI programming is done in Common Lisp. If you don’t know it, learn it. Learning a language is not learning to program, however; and AI programming involves some techniques quite different from those used for systems programming or for other applications. You can start by reading Abelson and Sussman’s Structure and Interpretation of Computer Programs and doing some of the exercises. That book isn’t about AI programming per se, but it teaches some of the same techniques. Then read the third edition of Winston and Horn’s Lisp book; it’s got a lot of neat AI programs in it. Ultimately, though, programming, not reading, is the best way to learn to program.

There is a lot of Lisp programming culture that is mostly learned by apprenticeship. Some people work well writing code together; it depends strongly on the personalities involved. Jump at opportunities to work directly with more experienced programmers. Or see if you can get one of them to critique your code. It’s also extremely useful to read other people’s code. Ask half a dozen senior grad students if you can get the source code for their programs. They’ll probably complain a bit, and make noises about how their coding style is just awful, and the program doesn’t really work, and then give you the code anyway. Then read it through carefully. This is time consuming; it can take as long to read and fully understand someone else’s code as it would take you to write it yourself, so figure on spending a couple of weeks spread over your first term or two doing this. You’ll learn a whole lot of nifty tricks you wouldn’t have thought of and that are not in any textbook. You’ll also learn how not to write code when you read pages of incomprehensible uncommented gibberish.

All the standard boring things they tell you in software engineering class are true of AI programming too. Comment your code. Use proper data abstraction unless there is a compelling reason not to. Segregate graphics from the rest of your code, so most of what you build is Common Lisp, hence portable. And so on.

Over your first couple years, you should write your own versions of a bunch of standard AI building blocks, such as

a truth maintenance system, a means-ends planner, a unification rule system, a few interpreters of various flavors, an optimizing compiler with flow analysis, a frame system with inheritance, several search methods, an explanation-based learner, whatever turns you on. You can write stripped-down but functional versions of these in a few days. Extending an existing real version is an equally powerful alternative. It’s only when you’ve written such things that you really understand them, with insight into when they are and aren’t useful, what the efficiency issues are, and so forth.

Unlike most other programmers, AI programmers rarely can borrow code from each other. (Vision code is an exception.) This is partly because AI programs rarely really work. (A lot of famous AI programs only worked on the three examples in the author’s thesis, though the field is less tolerant of this sloppiness than it once was.) The other reason is that AI programs are usually thrown together in a hurry without concern for maximum generality. Using Foobar’s “standard” rule interpreter may be very useful at first, and it will give you insight into what’s wrong if it doesn’t have quite the functionality you need, or that it’s got too much and so is too inefficient. You may be able to modify it, but remember that understanding someone else’s code is very time consuming. It’s sometimes better to write your own. This is where having done the half-dozen programming projects in the last paragraph becomes real handy. Eventually you get so you can design and implement a custom TMS algorithm (say) in an afternoon. (Then you’ll be debugging it on and off for the next six weeks, but that’s how it is.) Sometimes making a standard package work can turn into a thesis in itself.

Like papers, programs can be over-polished. Rewriting code till it’s perfect, making everything maximally abstract, writing macros and libraries, and playing with operating system internals has sucked many people out their theses and out of the field. (On the other hand, maybe that’s what you really wanted to be doing for a living anyway.)

A whole lot of people at MIT