But different LED colors result from different semiconductor compositions; it is sometimes possible to grow a thin layer of (for instance) red-LED GaAsP on a silicon wafer, but subsequently growing a green or blue (InGaN) wouldn't work (might not stick to the wafer, or would damage the underlying layers...)
Multicolored integrated circuits with three (or more) LED shades are theoretically possible. There's a LOT of bugs to work out, though.
Actually assembling an array from loose parts... is possible, but impractical. Onesies of three colors is a start, but a display has millions of pixels.