Basically, anyone who fought at the behest of the Catholic Church against "heretics" was considered a crusader, regardless of whose colors they wore. The Templars, Hospitallers, etc. were religious-military orders. If you joined an order, you usually sacrificed your own heraldry and wore that of your order. Think Balian in
Kingdom of Heaven versus Arn in...well,
Arn.
Both were crusaders, but Balian was not a member of an order and thus retained his own heraldry, whereas Arn was a member of the Templars and had to wear the Templar cross in place of his own arms (the "Azure, a lion rampant Or", or whatever it was).
Usually, however, a crusader would wear a cross somewhere on his person to signify that he was indeed a crusader. Like this fellow from the Westminster Psalter:
http://upload.wikimedia.org/wikipedia/commons/4/45/Westminster_Knight.jpgAt least that's how I understand it.