hey littleT,
i'm also in the SE USA, the southern most state, and I did an outdoor grow this past winter. the grow was great but i had a constant problem with pests - aphids, gnats, leaf miners and even some fungus (which i nipped in the bud, no pun intended). it was a real war zone but i won that campaign. i can't imagine trying an outdoor grow here in the summer.
have you had many problems with pests?