So he is pretty timid, and he really dislikes men. He allows you to pet his head and his neck but anywhere past his neck he lets out a little growl and runs away. He has not bitten me (yet). He gets along really well with my dogs, and you can tell he really REALLY wants to play with them but they don't know what to think of him yet

So my question is... how hard will it be to get his trust back with people? Is there anything I should be doing to help ease him into becoming more comfortable and friendly around me, or is that a lost cause given his age and how long he's been like this?
Also, he is NOT neutured... I do plan on neutering him - would this help as well?