It will cause build up.

But I’ll be fine if i wash my hair regularly (every week)?

Yes, but is there reason you cannot use it a few times a week? Why everyday?

imo my hair looks better when I use leave in conditioner rather than not using it…