The PICkit 2 'does what it says on the box', reliably programming the vast majority of PIC10Fxxx, PIC12Fxxx, PIC16Fxxx, PIC18Fxxx and PIC24Fxxx parts. It also supports basic debugging (register inspection, single step and single hardware breakpoint) on all parts that have native silicon support for ICD (a few PIC16, nearly all PIC18 and Pic24) Compatible Debug 'headers' are available for most of the rest. A few of the latest parts are not supported, but there is supposed to be a final update later this year and the firmware, stand-alone GUI and command line utility are open source.
The PICkit 3 is supposed to be the successor to the PICkit 2. It is still very buggy and I would NOT consider buying it as my sole programmer yet. When it is fully debugged it will probably be faster than a PICKit 2 and may have better features, but at the moment there is a strong suspicion it has serious hardware problems and the software lacks many features PICkit 2 users expect and is not exactly stable.
With either you need some way of connecting to your chosen PIC. You can 'roll your own' ICSP hook-up or buy a kit that includes a demo/development board.
Avoid the 'PICkit 2 Starter Kit' DV164120 as the supplied 'Low Pin Count Demo Board' for 8/14/20 pin mid-range PICs does NOT support any PIC that has built-in debugging support. The bundled PICkit 2 is of course ICD capable but the complete kit doesn't get you there. There are 18 pin and 28 pin DIP demo boards available that you can plug a debug capable PIC straight into but they aren't bundled as a kit and dont have a nice set of tutorials.
The 'PICkit 2 Debug Express' DV164121 is usable for debugging 'out the box' but the demo board has a soldered on surface mount PIC16F887, with a surface mount prototyping area and all thru-hole pads for off board connections at 2mm pitch :-( so can be a pain to hook up to your own breadboarded experimental circuits.
There is one other kit that rarely shows up if you are looking for a PICkit 2: the 'PICDEM Lab Development Kit' DM163035. From Microchip's site:
It is bundled with a PICkit 2. One of the supplied PICs is debug capable. The solderless breadboard area is large enough to take 28 or 40 pin PICs jumpered to the programming signals from one of the sockets so it will do to program all DIP pics the PICKit 2 supports.
You should also check out this link:
A PICkit 2 (or 3) compatible demo board with a ZIF socket for programming/testing all DIP PICs that the PICkit 2 supports up to 40 pins, on board regulators, 6 LEDS and I2C/SSP level translators. (It is definitely superior to Microchip's ZIF socket adapter for the ICD series which itself requires an adaptor to connect to a PICkit 2 or 3_ They also make an enhanced PICkit 2 clone with a very good reputation.