If you can remove the fan temporarily, turn it so that the blade is sideways. The heavier blade(s) will sag toward the bottom and stay there. You can then counterweight it on top. It will be in static balance when you can turn it to any position and it won't rotate on its own. It won't necessarily be in dynamic balance, but since it's probably a largely symmetrical arrangement and it runs at slow speeds, you're probably OK.
Don Mechanical Engineer Kansas City