Use a CF/MMC card. These have about the simplest interface to implement (there are actually two interface modes, with one being like IDE, and the other a simpler but slower synchronous serial interface). Elector magazine, did an article, with a circuit, and software for this, and dozens of similar interfaces are available for other micros (8051, Atmel, PIC etc.). Remember that to 'exchange data beween a PC and a microcontroller', the microcontroller, is going to have to handle the FAT filesystem, not just write the card as if it was a lump of memory. This will involve a significant amount of software, even if you limit it to a 'subset' file system, perhaps only supporting FAT12. With this in mid, there are small interfaces on the market, offering a serial interface to this, with the file system support already included, and there are similar modules to handle USB memory cards, which might well be a very 'easy' option. The Vinculum VDRIVE module for example, support async serial, or SPI interfacing, to a USB memory card.