When I looked into various modulation schemes (for digital signals, or at least conveying bits) about twenty years ago, I found many wonderful techniques claimed for bandwidth reduction. They all boiled down to filtering - either the individual bit stream(s) before modulation, or the modulated (subcarrier) signal afterwards, or combinations of both. Abbreviations often included an N or an M for "near" or "minimum".
Quite apart from the theoretical Shannon limits someone has already mentioned, there was (and still is) the matter of clock recovery: many of these papers often had "assuming perfect timing" hidden away somewhere (or even just implied). In a real practical system, where you're trying to convey information from one place to a distinct other, with no cheat wire round the back of the bench carrying the clock signal, you have to recover your clock timing - i. e. where the symbols start, end, and are to be sampled - from the actual received signal. With many of these filtered wonders, this is not practical!
Of course, digital signal processing has come on enormously since then (even twenty years ago I had to just about admit, though it pained me to do so, that it was possible to do better than Morse code, with _very_ complex algorithms!), such that it _is_ possible to do a lot better at clock recovery than it used to be, but the basic problem does still remain.