I see that you are in North Carolina---you would want to contact your local department of social services to receive a definition of "neglect".  Perhaps you can assess if your child is negatively affected by his father's alcohol problem and issues surrounding visitation.  If you have seen some changes in your child's moods or behaviors, a visit to a mental health counselor would be appropriate.  Your son will be able to work through any issues he may have with his therapist, and you may receive some support as well as you deal with this difficult situation. It might be a good idea to attack this now before school starts as counselors who treat children generally have more openings during summer vacation.  Best to you.