If you'll humor me, I'll explain fragmentation using a metaphor. Imagine that data is stored in books (a big stretch, I know :)), and that a hard drive is a library. When you [your OS] wants to read a book [program], you tear pages out and leave them lying around. When you want to add more pages [install more software], you throw the pages in the closest available position relative to where you're standing in the library. Pretty soon, it will start looking more like ground-zero than a library. This is where defragmentation comes in. Basically, it picks up all the loose pages [file fragments], puts them back in the books, and files them where they're supposed to be.
Now, it should be apparent to you at this point that this is not the most efficient way to read books. Leaving pages in the most convenient space relative to yourself may save a second or two at the time (or maybe not), but when you go back for that information later it takes a lot longer to get the same info. Pretty soon, you can actually start losing information!
Just like the library metaphor, there are much more ideal ways of handling your data than carelessly throwing it around. By simply keeping things organized in the first place, you can actually eliminate the need for defragmentation. The NTFS filesystem is indeed a step forward for Microsoft, as it handles data in a much more contiguous fashion than the older FAT filesystems, as well as being much more resistant to damage. However, as you have seen it still leaves much to be desired. There exist today at least a dozen filesystems which are technically superior to NTFS, most of which being open standards, but Microsoft refuses to implement any of them. The Reiser filesystem, for instance, is faster, more stable, more scalable, and more resistant to damage than the NTFS filesystem. Most filesystems today are self-maintaining, the only major major exception I can think of being NTFS.
Let me make it clear that my reason for relating this information is neither MS bashing nor Linux advocacy. I simply wanted to make it known that severe filesystem fragmentation is not a technical problem that all computing in general suffers from, merely one software vendor in particular.
-AT