That's not a simple question.
Decide what language you want to use to program it. If you want to use something simple like a basic-like language, you'll want to consider the cost of the development language tools. The harder the language is, the cheaper the tools.
Do some googling to see how much free sample code and hardware design is available for your choice. If you can't find lots of relevant stuff, reconsider your choice.
Beware of free tools that let you program half the chip memory. You want to use it all, it'll cost you.
Take a good hard look at the high level commands of the language. You can do anything with C...and the GCC tools are free. But, do you want to spend your time figuring out how to format a text string in a subset of C or sending that text string formatted in a variety of ways with one command? Yes, you can probably find a text formatting "include file". I find C to be a stressful programming language. Any language that requires you to determine the need and find the correct include file just so you can concatenate two strings is too complex. Stuff in C is pretty simple the second time you do it...assuming your memory is excellent. I find that FIRST time very distressing. And it's hard to get help. You'll get lots of flak from people who've already done it once. You're an idiot because "everybody knows that"... except those of us who haven't done it once. Since "everybody knows that", it was omitted from the step-by-step instructions that didn't work for you.
Do you want to build a complex breadboard for development, the make a circuit board for the finished project? If so, bare chips are what you want.
If you want pre-built hardware, you might want to start with something like an Arduino. More cost, simpler deployment.
I find a bootloader to be an essential feature. I've busted more stuff trying to plug/unplug chips to be reprogrammed than all other faults combined. But you may still need a programmer to put in the bootloader.
For me, the sweet spot is PIC16F877A with PicBasic Pro and tinybootloader. I do all my I/O through the serial port. If I want USB, I plug on a USB/serial adapter. If I want wireless, I plug on a Bluetooth/serial adapter. If I want GPIB, I plug on a GPIB/serial adapter. If I want infrared, I plug on a serial/IR adapter. If I want a GUI, I plug on a Palm IIIC and use that for the user interface over the serial port. The palm also does the "heavy lifting" like floating point math, buffering, formatting, graphics, touch screen, etc. With the Bluetooth/Serial adapter, you can use a more modern palm with bluetooth and more horsepower. A 25-cent Palm PDA can save you a LOT of grief at the embedded processor end of the wire.
For the 16F877 (won't work on the 877A) there's a resident basic interpreter. Just plug on a terminal and start typing. Yep, you can do that over Bluetooth too. You'll want to modify the source code slightly so it can be installed with the bootloader.
But I got the compiler for a buck at a garage sale and Microchip used to give away sample chips. Not sure I'd go that way today.
TI has some cheap plug-on processor boards for the EZ430 usb development dongle. I started this direction, but decided that the learning curve for a new system outweighed any benefits over the current PIC system. Looks like they've tripled the price of the boards. $10 each is not as attractive.
Arduino-like systems seem to be the most popular today.