Asking this on a VHDL newsgroup may not be the best idea... try comp.arch.fpga (given that you are thinking of using Xilinx devices) - I'll cross post this there...
Yes.
There are many more questions to be answered now...
For example.. How fast do you want to do it? Are you going to use internal or external memory to store your matrices? (Are there devices with 12MBit of internal BRAM...?)
My compilation's finished now, so I'm back off to the lab now...
Cheers, Martin