It is correct that the nonlinear voltage-to-brightness transfer function will produce some brightening where chrominance amplitude is high. However, the magnitude of the effect is small enough in typical situations that observers do not find it objectionable--in fact, many prefer the increase in average brightness.
What many dislike when viewing a color video signal on a monochrome monitor is the strong dot pattern in saturated areas--more a problem when viewing close, as with a computer monitor.
Some later model B&W sets incorporated "color traps" to reduce the chroma amplitude, but most manufacturers considered this a luxury. ;-)