USB to serial port adapters exist, they are not expensive.
Flash to store the data in whether flash chips, CF card or other type is perfectly OK, as long as the logger can address the device either directly or by keeping track of which sector it is on.
Your own internal flash chips sodler on the board reduces reliability problems of connectors or wrong type of CF card installed.
Alternative 1 =============
As myself and others said before look at FTDI USBserial adapters, then have the flash inside the device log the data via RS232 at 4800baud.
Then when you need to talk to the PC use the USB to RS232 adapter at very much higher baud rate (115000 or higher should be possible on most micros) Use a simple application on the PC to read the device using a USB pseudo serial port.
This has the advantage of you DON'T have to add USB hardware and software as well as the RS232, and you only need ONE serial port. The flash can be soldered on board a smaller and more portable and SEALABLE weatherproof device.
Also you don't have to get your USB checked on hundreds of USB combinations and buy IDs etc..
Alternative 2 =============
Using a CF card in a PC without a file system and doing raw read/writes has a basic problem. Windows allows raw read/writes when a physical drive exists and has a drive letter. CF card does not have a drive letter until the Plug and Play from detecting the CF enumerates it and does an auto mount including mount a filesystem. Until you either bypass that mounting of file system or only allow format of device options that windows has on automounting of a device, you wont be able to do raw read/write on a drive that may not yet exist to the system. Floppies and other devices can do raw read/write because the drive physically exists and exists as far as system is concerned.
As someone else has said know what the format of the device is and skip the filesystem blocks is one way, but bear in mind as the CF cards get larger they maybe formatted as FAT16 or FAT32 depending on size of the CF card. That in turn will affect your coding.
CF cards and connectors means all sorts of other reliability issues of the connectors, CF card brands supported, mechanical considerations for sealing and durability.
Those are the basic choices, decide what you want from the product as a whole in usability, reliability and other aspects, using the same serial port in playback mode means less openings and connections to deal with.