You can remove the collars from the rest of your characters at the Seeker Camp after escaping Fort Joy and rescuing Gareth.
A tinker in Fort Joy can remove your primary character's Source Collar after doing some combat trial underground (enter through hatch near where Griff is standing).
Removing your own and your party's Source Collars:.
Magister Waters won't let you touch this collar so you have to kill him if you want to pick it up. One of the many strengths that Divinity: Original Sin 2 has as a game is its ability to bring variety to traditional character classes from the medieval.
There is one near the body of Finn on the Lower Deck of The Hold.
This collar does not provide any buff, debuff, or protection.
In its inactive state, the collar appears to be nothing more than common metal.