Thanks to @execUTEioner who helped me tweak my server email settings yesterday. The spam score has been greatly improved from one which most mailboxes would reject, to one most will accept. There’s one more small thing which is not a huge issue, so as it sits now outgoing emails should reach more users.
There’s still the problem with the site’s IP address being on one email blacklist which is used by crapcast and I think maybe msn and hotmail. SO those emails may still not make it through. But, I’m working on getting that blacklist entry removed.