Hi Paul,
I don't mean to pick on you, but... you really should become more familiar with VNA measurements in general. You should be able to find documentation on the web; try the Agilent site for example, and look for VNA ap notes. I downloaded some recently, so I'm pretty sure they are there. A VNA is a wonderful piece of test equipment, but it won't give you answers you can trust if you don't understand it. I would trust what they tell you (or similar from R&S or other reputable manufacturer) more than I would trust replies here to be not only accurate but complete.
For example, you should understand that your dummy load could be considered a two-port device where S21 and S12 will be very low--hopefully vanishingly close to zero. If you connect it to port 1, you should get a valid indication of S11. You should expect S22 to be unity or very close to it, if that port is left open or shorted. (But why would you even bother to ask for S22, or S21, or S12?)
But there's another problem here. HOW do you calibrate your VNA?? You ideally should have a precision short, a precision open, and a precision load--best if you have calibration data for the loads. If you measure your load, how will you know whether an imperfect S11 is due to the load, or to the VNA? Of course, it will be to some extent both, but it would be nice to know that the VNA is accurate to within some tolerance.
Cheers, Tom