I'm a CS student, and honestly, I don't understand Knuth's books

Question

I stumbled upon this quote from Bill Gates: "You should definitely send me a resume if you can read the whole thing." He was talking about The Art of Programming books. So I was pretty curious and want to read it all. But honestly, I don't understand it.

I'm really not that intellectual. So this should be the reason why I can't understand it, but I am eager to learn. I'm currently reading Volume 1 about fundamental algorithms. Are there any books out there that are friendly for novices/slow people like me, which would help to build up my knowledge so that I can read Knuth's book with ease in the future?

Answer 1

Be sure to read all of Gates's quote including this:

"It took incredible discipline, and several months, for me to read it. I studied 20 pages, put it away for a week, and came back for another 20 pages. If somebody is so brash that they think they know everything, Knuth will help them understand that the world is deep and complicated."

They are not easy books and they're not intended to be. Remember that one of Knuth's goals was to bring mathematical rigor to computer science. That's great if you want to prove something about an algorithm, but not-so-great if you just want to know how it works.

Michael Dorfman has a few good tips for reading the books in his answer to the (now deleted) question on Stackoverflow about What Can I Get From Reading the Lot?. If you don't have 10k rep, you can still view the question and his answer on the wayback machine.

What will you get from reading the lot? An excellent foundation in Computer Science. You'll understand how computers operate, from logic gates on up through compilers. You'll think about problems you never really knew were problems (i.e., what is the fastest way to multiply?) and see algorithmic connections between things that you never thought were related (riverbeds, RNA and nested parentheses, for example.)

I completely disagree with the folks who say "build software instead of reading about building software"-- there is a difference between the disciplines of Software Engineering and Computer Science. TAOCP is about the latter.

If you haven't yet begun, I have a few recommendations.

First, you may want to begin with Volume 4. It's some exciting material, very up-to-date, and Knuth's sense of humor shines through. Plus, there are videos available (on the Stanford SPCD site, or the Stanford iTunes) where Knuth discusses various sections. These videos are highly recommended. Fascicles 0, 1, 2, 3 and 4 of Volume 4 are available as separate paperbacks. Together, the published V4 material is larger than any of the first 3 volumes, but sliced up into bite-sized treats. (I wonder if Volumes 1-3 would seem less terrifying to people if each volume had been published in single-chapter paperback form...)

Depending on your math background, I'd recommend you skim Chapter 1 the first time through, and return to it as needed. In fact, you'll probably want to read each section (at least) twice-- quickly the first time, just to get the intuition and the gist of the arguments, and then slowly, carefully, understanding each step.

Be sure to read Volume 1, Fascicle 1 on MMIX instead of the old sections on MIX. MMIX is better in many ways, and you'd be better off converting the MIX in the text to MMIX as you go along than trying to straddle both worlds.

A general rule: don't skip the exercises. There's a lot of good material in the questions (and answers). Do as many exercises as you can; but read them all (and read the answers, once you've taken a shot at the problem, or decided to give it a pass).

Finally, if you really catch the bug: read the index. Lots of great jokes hidden away there.

Naturally, StackOverflow would be a good place to post specific questions on the text, if they should happen to arise....

For other resources, I've found that browsing the syllabuses of well-regarded computer science schools is useful. For example, textbooks for beginning algorithm classes:

• Stanford uses Algorithm Design
• Carnegie Mellon uses Data Structures and Algorithm Analysis in Java
• MIT uses Introduction to Algorithms

Answer 2

Even I think Knuth's book is a bit advanced and difficult to understand. Those books are definitely for research level algorithmists IMHO.

So are there any books out there that are friendly for novices/slow people like me?

Introduction to Algorithms by CLRS is much simpler.

EDIT :

Still if you want to read Knuth's book you should first go through Concrete Mathematics. Knuth wants his students to be aware of the basic mathematical portion of algorithms analysis.

Answer 3

Knuth is the most revered, quoted, talked about, and highly respected computer science author in history. His books adorn the bookshelves of all serious software developers, and are referred to with the same level of respect people give the Bible and Art of War.

I've even heard that some people have actually read portions of Knuth's books.

Most people just intend to.

_{Personally, I'm saving them for my retirement}

Answer 4

Knuth's books changed the field of algorithms forever. He himself has said that '2 pages in my book is somebody's entire career work' and that his books were difficult to read. The book contains condensed material from years of work in Computer Science.

You should not feel bad if you cannot understand it.

As Prasoon said, CLRS is a simpler book to read.

You also have Algorithms by Rajasekaran, Sahni et al which is easy to understand.

Answer 5

When I first graduated I picked up the first three volumes of TAOCP as a graduation gift to myself and attempted to read them straight through. Never managed it. These days I've made it through maybe 1/3rd of the first three volumes (thought in no particular order). The material is definitely dense, but there are three tips I've learned that helped a lot.

First, don't try to read from cover to cover. TAOCP is really as much of a reference work as anything, and I've found it's best to read a section when it's relevant to a problem you're trying to solve. Like many things in the world, understanding the solutions is much easier once you've encountered the problems they are trying to solve.

Next, that flowchart in the front of the book, it's not just a bit of humor but actually quite a useful tip. Read the sections you're working on iteratively, starting first with just the overarching concepts and then gently getting deeper into the math.

Finally, keep some good old fashioned paper and a pencil handy to work through the algorithms as they are described, and work through a few of the easy problems. It goes a long way to help re-enforce what you're reading.

Answer 6

Don't worry, most people don't understand The Art of Computer Programming (TAOCP). So don't think yourself as slow or a novice for not understanding-- you're just like the other 99.99% of us who don't get it.

You're pretty ambitious if you want to get to the level where you can read The TAOCP with ease. I myself only have paged through the books before putting them away. There's probably only a handful of people on this planet that understand TAOCP.

Check out the post: Books Programmers Don't Really Read by Bill the Lizard.

There are plenty of other books listed there that are quite readable, understandable, and you can benefit from right away.

I personally like:

• The Pragmatic Programmer
• The Mythical Man Month
• Code Complete

Answer 7

I stumbled this quote from Bill Gates: "You should definitely send me a resume if you can read the whole thing." He was talking about The Art of Programming books.. So I was pretty curious and want to read it all but honestly, I don't understand it at all.. I'm really not that highly intellectual being.. So this should be the reason why I can't understand it, but I am eager to learn.. I'm currently reading volume 1 about fundamental algo.. So is there any books out there that are friendly to novice/slow people like me? So I can build up myself and hopefully in the future I can read Knuth's book at ease..

if you define yourself as not a highly intellectual being then you are setting yourself to low expectations. You got to break that mentality if you want to do something worthwhile the trouble. There should be no doubt in your mind that you can achieve something. Also, achieving it doesn't mean that you will achieve it easily.

Things worth pursuing are the ones that are difficult... and that's not a cliche. In software, engineering, on in life in general, if you want to achieve something, you got to go for the things that are hard, the things people avoid, and not settle for the lowest common denominators of things.

First, it is not clear what your CS background is. Knuth's book require a degree of maturity. Few people with a CS degree can go through it with ease. I would not expect a CS student that has just finished his first course in algorithms to actually be able to go through even a single of Knuth's book. The maturity needed to get it is just not there, and that has nothing to do with the student's mental capacity.

You need to have your basic of algorithms down cold and clear, and you need to have a fair amount of programming (work and/or scholastic) under your belt - I'd say, 40 credits on programming at least. You also need to have your CS mathematics on firm ground.

You can't go far ahead without having a good grasp of discrete mathematics (and possibly theory of computation.)

It's not that you will need that knowledge to work on Knuth's problems, but you need a maturity to be able to go through that type of material.

First pick one book and one book only (CLRS' book as previously suggested), and work it from start to finish. When possible do programs implementing the algorithms. Don't use Java or C#, not even C++. Go to bare bones C and get the feeling of building things from bare-bone metal scraps.

Also get Knuth's book on "Concrete Mathematics" if you have not done a course in discrete mathematics and computation theory. Would be good for you to go through that book as well.

Then tackle Knuth's encyclopedia, one tome, one chapter at a time. Don't go to another chapter without having a good grasp of the first one.

I would suggest you go through volume I (fundamental algorithms) first, then volume III (searching and sorting). Those should be your immediate goals. Then, later (much later), tackle volume IV (Combinatorial Algorithms) and then Volume II (Semi-Numerical Algorithms.)

Don't feel bad if you don't get it at first. I've been trying to get through volume I and III for years (10 years now).

And you should not put so much weight on it either. Don't do it to prove something to somebody or yourself. Do it because you are intellectually interested in doing so. You can become proficient in algorithms by simply using the CLRS' book (or any of the fine grad-level books out there.)

Be pragmatic and give yourself a break. Treat going through Knuth's book as a long-term personal ambition, not as an immediate proof that you are CS material ;)

There are other more important things (career-wise) to kill yourself for ;)

Answer 8

Before even starting on Knuth I had to pace my way through four different books. The first two are the Sedgewick books on Algorithms. These overview most algorithms and data structures in an actual implemented form, so you can see what they are and how they work. These books come in different language versions - I read the ones in C, but iirc they were originally written in Pascal, and there are C++ and Java versions out there.

After that I worked through a good portion of the Cormen book on Algorithms and used Sedgewick and Flajolet's Introduction to the Analysis of Algorithms as a complimentary text, as it is more in the vein of Knuth's mathematical rigor than the Cormen book. I still have yet to finish either of these, mostly cherry-picking the parts that I feel I need.

After reading those, and getting a degree in Mathematics I can read through some of TAOCP, but it is a difficult read. That isn't to say it is not useful. TAOCP are some of the greatest algorithm reference manuals around, but thinking you can use them to "fully" understand anything is somewhat problematic.