Should I give up visitation rights and possibly terminate my own parental rights?

I am trying to free myself from an abusive partner. I have two children of my own and together we have one child, who is 4. He has a disability,dyspraxia which involves speech impairment as well as emotional problems. He has a vocabulary level of a 19 month old. Since I have left the relationship my abuser has kept our son. I have only been allowed to see him 4 out of the 6 court required time. He has made allegations of ual abuse against one of my children towards the 4 year old and is trying to have me charged with simple ault on a child. He tries to fight with me on a regular basis about these "occurrences" both being false and lies. I can not afford an attorney and so far have been unable to convince the judge of his abusive ways. I do not want to go to jail, I do not want to put the 4 year old through anymore turmoil, and fear the ugliness that is coming. Any suggestions?