Well, on both AMD and Intel chips, you can overclock them with the FSB. However, the Intel P4/Celeron's tend to scale very well here.
To get a very high FSB with the Athlon, you'll need to drop the multiplier, but you'll usually need to connect the L1 bridges on your CPU. (same goes for the Duron) Starting with the Athlon XP (Palimino and Thoroughbred cores), AMD made a laser-cut trench through the bridges to prevent the easy L1 pencil trick.
Before I go any further, lemme explain what the "pencil trick" is. All you do is connect the dots noted as L1 on the processor packaging with a sharp, graphite (or lead) pencil. Personally, a mechanical pencil helps here. This works on any Socket A Duron/Athlon using the "Thunderbird" core, and these do not have any XP lable on them (nor MP, which denotes the multiprocessing capable version of the Palimino/Thoroughbred core). From 1.2 to 1.4GHz, the Thunderbird Athlons came with the L1 bridges uncut. I have a 1.4 Thunderbird, and it did come with the L1 bridges intact.
Anyway, starting with the Palimino's, the laser-cut trench on the L1 and other sets of bridges to keep people from reconnecting them and playing with the multiplier. This didn't stop people however. They got around it by filling the trenches and connecting the bridges. Most people use rear-window defogger repair paint.
Cooling depends on the stock speed of the CPU and what cooler you're starting out with. The Pentium 4 stock cooler is really pretty good, and unless you're a die-hard overclocker (and even then) the stock cooler is more than enough.
For the Athlon's, the stock cooler is good enough. If you want to pursue overclocking, look at getting a cooler that has a copperbase, or at the very least, a copper insert in the base. Expect to pay at least $20-30 online for a good cooler, and then some extra for a fan, depending on how big and how fast it is.
Other factors that can affect the cooling of the CPU is the airflow in your case and the thermal compound you use, Arctic Silver II being some of the best available.