Is this retaliation?

I recently quit my job as a Director of Sales for a Technology company and never signed an NDA (Non-Compete Agreement). I took a position with a partner of my last employer (with the blessings of the President) and now find myself in what I believe is retaliation situation. He continues to threaten me with letters and phone calls about interfering with their existing customers and luring clients away. He even went as far as to send a copy of the letter from his attorney to the President of my new company. This letter was addressed to me, not the company I work for. None of the customers I have contacted have contracts with my previous employer nor had they done business with them while I was employed. He is claiming that my conduct is "illegal in California and exposes me to substantial monetary damages". If I have not signed an NDA of any kind and he has not worked with these clients why would his attorney threaten me this way. They are also asking me to return a 10,000 draw that I was given as an incentive to join the company. I was with the company for 8 months and no where in my offer letter does it say this is a recoverable draw nor did they ask for this money back while I was employed.