Keep doing what you are doing. There is something very gratifying about planting something in the beginning of summer and harvesting some produce along the way. I share many of the same concerns as you with my skill set, but I continue with it year after year. Cheers from one New Yorker/teacher to another!