He can take it off you. The question is what will anyone do about it. It gets tricky. I was in the opposite situation. I had the sole use of the car and it was in my name. Then we separated and I left the house, initially on a temporary basis, and I stupidly thought we'd be adults about it and discuss our property split later on. So I left the car with my ex as she would need it to transport the children. She kept the car, a few weeks later took out an intervention order (DVO) on me, and I never touched the car again. She still has it 2 years later and hasn't wanted to discuss a property settlement, mainly because it suits her to maintain the status quo where she keeps everything.
Police told me if I tried to recover it, it would be considered theft (that's very arguable legally, IMO, except that I guess I would have to trespass to get to it) but my family law lawyer also told me that since we now have parenting matters ongoing, it would look very unfavourably on me to try to recover the car anyway. So I got screwed. I guess what it comes down to is this... If you have an agreement that you keep the car, it wouldn't look good if he took the car. If you have the car to transport your children and you have primary care of them, even better. But there's probably nothing legally preventing him from taking it other than that it would look bad.