Whoever is not with me is against me, and whoever does not gather with me scatters.
Qui non est mecum, contra me est: et qui non colligit mecum, dispergit.
This verse is often contrasted with Luke 9:50, which seems to say the reverse:
Whoever is not against you is for you.
Qui enim non est adversum vos, pro vobis est.
Luke 9:50 (or Mark 9:40)
I used to wrestle with this, but it seems to me tonight to be two parts of the same teaching - not two contradictory statements. Jesus never says "Whoever is not against me is for me." He says, about himself, whoever is not with me is against me. It's notable that he's speaking of Satan in this passage as "not with me". But he's speaking of the Church in "Whoever is not against you" (which pronoun is plural and so should be translated "against all y'all").
What this means to my eyes is that we cannot call Christians those who hold only lukewarm ideas of Christ: if they are not with him, they are against him. But the Church can call these same people friends or Ecclesial Communities, if they don't hinder us in our evangelism. They can work with the Church in our outreach, our social ministry. But we cannot afford to confuse common, if you will, political goals, with our God-revealed telos or right-ending. The purpose of our actions must always be ad astra, or to the stars. The purpose of our politics is not earthly: the Church does nothing that cannot be for the salvation of others.
Jesus, being God, reveals the telos, the end point of all creation in time. Jesus, being man, reveals the telos of human nature in divinity. Whoever is not with him - fully, wholeheartedly, committedly - is against him. If you're not willing to give all and die, go home. By the same token, if you're willing to put up with us, with our insanity, with our prolife marches, our teachings on sex, our insistence that there is one right way upwards, then come to the party! Even if you think we're making all the stuff up, you're welcome. But if you just want us to pretend to be a social organization, a political club, or some kind of fancy-dress cheerleading squad for your partisan politics, we will have to decline.