Re "So the argument must be falling down somewhere, probably because I can conceive of Him not existing.":
So the "Him" you can conceive as not existing is clearly NOT the Him whose non-existence is inconceivable! The God you conceive might not exist is an image that you've constructed in your imagination based on your Sunday School lessons, so is essentially an *idol* - a false god. It is good news that you see that idols can't exist. The more idols you dismiss the closer you come to the real God that lies beyond your or anyone else's conceptions. The 14th-century theologian Meister Eckhart made the same point: "The more they curse God the more they praise Him!" Re "Seems reasonable to me that God would have a strong moral sense, stronger than mine even, and that he wouldn't like to see people suffer.": The Godhead doesn't have a strong moral sense. It is the crassest anthropomorphism to imagine otherwise. (It's another category error!) But we humans have a moral sense ("The soul is naturally Christian" - Tertullian, third century) so we should encourage that moral sense to flourish in the same way that a gardener encourages a flower to bloom and emit its fragrance.