How can a judge order "no contact" between my boyfriend and daughter in divorce case?

my x is a very mad individual because i left him. We live in a very small town where the good ol boy system is alive and well and the presiding judge is a "friend of the family" to the X. I lost custody of my youngest daughter to him but i also have 2 other children from prior marriages the continue to live with me. Now the judge has added a clause to the divorce stating that our daughter can not be in the presence of my boyfriend. I do not understand how this brcause he in no pedifile or anything. What can we do because this is really hard on how we are living...when she stays with me my boyfriend can't come around or be with us doing anything. Seems like this is a violation of my daughter's and my boyfriend's civil rights somehow? My x is very spiteful and has his "family friends" on his side! HELP...