Leslie's essay is a strong argument for agnosticism. The occurring idea is that knowledge has its limits. He defines a Gnostic as someone who "holds that we can attain truths not capable of verification, and not need needing verification, by actual experiment or observation." (page 98). This seems to include any theists, atheist or even historian or lawyer.
It is interesting that in the Portable Atheist there is such a strong argument for agnosticism. The article before this has Darwin declaring himself an agnostic. Leslie says that atheism is "a rare phase of opinion" (page 97) and is baffled by both the atheist and orthodox making claims to Spinoza (he has already made an appearance in this book). He also specifically points out that Spinoza and Hume do not agree with each other about miracles (Hume has also made two appearances in this book, once about miracles), which raises questions as to why these authors are placed in the same book about Atheism.
For some reason unclear by experiment or observation, Leslie sees that God is can either be loving or just, but it's not possible for God to be both. He does mention that Jonathan Edwards theology is "logical, but not consoling" so passes over that (page 107). Which is interesting as Leslie says this about us all:
We are a company of ignorant beings, feeling our way through mists and darkness, learning only by incessantly repeated blunders, obtaining a glimmering of truth by falling into every conceivable error, dimly discerning light enough for our daily needs, but hopelessly differing whenever we attempt to describe the ultimate origin or end of our paths;... (page 110)
Leslie points out that throughout history philosophers don't agree with each other and then concludes that there is no certainty (except that one). He even says "that facts are what we feel them to be... and facts are just as open to one school of thought as to another." (page 109) Which then makes it hard for him to argue that his reasoning is coming from anything other than what he is feeling. Since the main stumbling block for Leslie is that no one can agree on the big issues, perhaps he should just go with what most people agree on, which would make him at least a theist considering that he admitted that atheists are quite rare.
Leslie struggles with natural revelation, and correctly points out that two people can see the same thing in nature and one may become an atheist and the other a theist. He is right to argue for something more certain that just the natural world as evidence for God. If only God did speak to us through some means that we could understand and learn about Him. After all, we can only learn a little bit about a builder by looking at what they build; what we really need is that builder to talk to us and tell us (reveal) what his is like, so we can get to know him better...
You can read all of An Agnostic's Apology here.