I've never been a member of any "party" (or professional organization, etc.).
Be advised, though, that the parties have rules as to what you can and can't do as a member/nonmember. E.g., I can't vote in any of the national primaries as I'm not a member of any of those parties (makes sense).
OTOH, there are some (local?) elections in which I am asked by the polling staff as to which ballot I'd prefer to vote -- a choice that is NOT available to declared affiliates of one party or another.
Likewise, the solicitations, junk mail, etc. vary depending on whether you've declared a party or are a "potential swing voter".
(We have LOTS of elections, here -- at least one or two each year. So, your choice can have significant consequences)