I've often pondered the same thing. My first online researches revealed only a host of Christian conspiracy-theories, all of whom agree that Tolkien was a puppet of the Illuminati, and wrote his books (which deal with such evils as sorcery and witchcraft) in order to delude the Christian world at large. I didn't know this, but the devil gave him the inpiration to write TLOTR in order to undermine Christianity. Also, did you know that Oxford is controlled by the Illuminati, and the school is a means by which they exert their sinister influence on impressionable minds?
One thing, however, is certain: he was well read in Runes, and ancient Norse texts, etc. There are a great deal of spells and incantations in such works, and he drew a great deal of inspiration from their study. He also expressed an interest in Egyptian mythology and the ancient texts of the East (the oracles of Zoroaster, etc.)--so it would be safe to assume that a great deal of Magical lore was incorporated in his books. The Nordic runes in particular deal with many spells, incantations, and so forth.
If he was a member of an Order, he did well to mask his true leanings behind his Catholisim.
As for myself, I have little doubt that his works were written with full knowledge of Qabalah. He says himself that they have no allegorical meaning at all, but then again, in our world of misconception and prejudice, one does not always reveal the truth of things to the profane.
One also wonders about comparisons to Wagner's "Ring Cycle," for Wagner was in close contact with high-ranking Masons. (Also he is listed as a former member by the OTO.)
Edit: Ah, a little research shows that, while he was never admitted (due to his "immoral" lifestyle), he applied to a Masonic order, and often expressed a desire to join the Masonic fraternity. Many of his close friends and associates were Masons, so he most likely picked up a lot of Masonic symbolism from their correspondence.
Quite interesting, as Tolkien and Wagner are two of my favorite artists. One recalls, also, that Nietzche was a close friend of Wagner's, and Nietzche had a vast knowledge of ancient lore: the Oracles of Zoroaster, Greek rites and Magick, and Norse mythology.
Edit the second: consider, also, that the hobbits and orcs may represent elementals.
And I just recalled that Tolkien was well read in the Egyptian Book of the Dead, which is rife with spells and incantations of a Magical nature. He also said (I paraphrase) that the inhabitants of Middle Earth are best conceived in Egyptian terms, such as those figures found in the BOTD. (Funny, I always thought of them as rather like Vikings and Knights, etc.
I'll do a little more research on the subject, and perhaps we could compare the Lord of the Rings with Qabalistic archetypes, if nobody minds.