I suppose I ought to add “and literary history” to the title, since this rant is also about that.
Confession time (or not, since I know some people already know this): I’m an English major, currently in graduate school and working towards my Ph.D. My main area of specialization is nineteenth-century British literature, especially Victorian poetry, and I have a special interest in Swinburne’s poetry and themes of atheism and agnosticism played out in poetry. So a large part of this rant is influenced by English geekery as well as by ordinary reading.
On to various ways to use literature—and especially ways in which fantasy can use it better than any other genre.

1) Retelling.

The most obvious candidate here is fairy tales. I wish it wasn’t. I mostly dislike the omniscient voice, and I have almost never liked archetypal characters, moral didacticism, and stories whose plots I already know. So most retold fairy tales and I don’t get along—especially when the author spends the whole of the story going, “Look at me! I mention sex! I’m edgy!” I have many of the same problems with other retellings, such as Arthurian and Robin Hood ones, unless the author is doing something spectacularly different.
But, my fucking gods, the canon of literature is not limited to fairy tales and English legend. There are tons and tons of neat, readable books out there, both fiction and nonfiction. And thanks to the Internet, you can find them and read them. Perhaps one of them will speak to you, and you’ll find yourself inspired to wrench a mainstream story sideways into fantasy, or write the fantastic version of a Victorian conduct book (the really quite hilariously cheesy things that Victorian English women were supposed to read and follow), or reimagine the common features of a story from the perspective of someone whom the original author mentioned only in passing.
I suppose I can’t spend much time on this, because it’s obvious, really, and many authors have done a retelling at one time or another. I do wish that more people thought to do something different with it, such as add in incidents that didn’t occur in the original story and deepen the characterization. Yes, a retelling is always going to be familiar in some respects, but what makes one outstanding is the sudden touch of the startlingly or painfully new in the familiar setting—especially when the sudden touch throws the whole comforting confines of the tale into doubt.

2) Fictional laws of the universe.

Imagine that you are reading Robert Browning’s Porphyria’s Lover, which happens to tell the story of a rather deranged “romance” from inside the deranged person’s head. Now, you could retell the story of the poem from Porphyria’s point of view, or you could attempt to write something that would recapture the tone of deep madness without following the exact outline of the story (see point 3). But there’s a middle road, too. What would happen if you assumed that the way the lover saw the world was the world as it is, and tried to write a story set in a universe in which the laws of the poem took the place of natural laws?
Yes, deeply weird in the case of this poem. I’ll choose a slightly less alarming example. Enoch Soames by Max Beerbohm is a kind-of fantasy short story in and of itself (though you could argue that some of the elements are science fictional, and certainly some are satire of contemporary poets and writers, and some people might say that since the Devil is involved, the whole thing moves out of fantasy and into the category of religious fiction). But I’d really like to see someone write a story like this—only set in a fantasy world with its own literary traditions, as intricately developed and intertwined as the ones that Beerbohm is mocking here. It’s quite an ambitious project, given that you’d have to let worldbuilding as well as characterization and plot shine through the pages. I don’t care. I want someone to write a story like this.
Setting a story inside a story—even a story distorted beyond all recognition by the fantasist’s own ideas and sensibilities—is something fantasy will do better than any other genre. It doesn’t have to turn into allegory or satire. It can be an attempt to extrapolate from the original work just to see what the hell happens.

3) Adaptation.

Rather than retelling the story or unfolding the stories hidden in its shadows, this term—as I use it—means striving to achieve something of the tone of the original story as a whole. The most common way to do this is through style and diction. If a story’s style immediately leads a reader’s mind to the King James Bible, or to Sherlock Holmes, or to H. P. Lovecraft, then the author is most likely trying for that effect, and for a particular reason. (Of course, if the author isn’t trying for that effect, then it may be a case of having read too much of one particular kind of literature and needing to expand one’s reading field).
Think of subtler ways than simple style and word choice. Reread your favorite poem, or short story, or novella, or novel, or trilogy, or never-ending series. How does it achieve the tone that you get out of it—whether that’s passionate sadness or exultant joy or bloody-mindedness towards the whole world in general? (I am of the opinion that the many, many people who want to write like Tolkien would be better off if they strove for a tone of passionate sadness expressed in their own manner, rather than borrowing bits of furniture wholesale).
One way to do it is to think of what attitude to the world that the work expresses, of course. Hard-boiled detective books can have all the trappings and still not seem like stories in that genre unless they also invoke that dusty, worn-metal, world-weary attitude. On an individual level, I would really, really like to read a book that touched the whole tone of calm despair present in Swinburne’s The Garden of Proserpine, or the fierce and degraded exultation of Dolores, or the disturbing (in just about every way possible) mystery of The Leper, or the fierce, scornful atheism of Before a Crucifix. But then, I am Swinburne-obsessed.
Tone, color, shading, attitude—all names for the same thing, in the end, the invocation of a particular effect from the work in question. Bring it up, work with it, and then move on from it. I think one of the easiest ways to get into a rut is just to keep repeating and reworking the same tone over and over. A writer might not be able to escape from the themes that fascinate her, at least completely, but she can take control of their expression.

4) Allusion.

Another very common way for fantasy to function, though, once again, authors tend to stick close to a small garden of works. Fairy tales, the Bible, and Shakespeare are all ones I’ve seen invoked fairly often. I’ve also seen two different books make Kubla Khan a piece of the plot in some way: Greg Bear’s “Songs of Earth and Power” duology, which treats it as magical (and also has a weird alternate Earth and weird-ass Sidhe, and is Good), and Douglas Adams’s Dirk Gently novels. So while familiarity might be a criterion for allusion, there’s also the case to be made for wild and sweet and strange power. And there’s a lot out there that has its own power, but which many people don’t invoke, even when they’re by familiar authors.
To make unfamiliar allusions work, I think, the author has to allude to them in the manner of a meaning beneath the surface of the story, rather than an in-joke. The story still has to be good and to have power and to be well-written even if the reader isn’t familiar with the work in question. If the entire point of the story depends on the reader understanding exactly who everyone is, it’s a problem. But here’s where fantasy helps again: the writer can make use of the figures and objects and tone of the unfamiliar story in the same way as ambient or mystical magic, making the background shine even as it’s not explained. If the reader happens to catch a hint of explanation, then that’s an added bonus.
I think this is probably the major reason that I like Kay’s Sarantine Mosaic as much as I do. I hadn’t read Yeats’s Byzantine poems when I read it, and yet he invoked them in so many allusions, both explained and unexplained, that I had the sense of an added layer of mystery moving along beneath the surface, hushed and deep. When I did read the poems, seeing what Kay had used and what he hadn’t used added an extra layer, in turn, to my understanding of the books. This was rather neat.
A couple less familiar works that have stuck in my head since I read them: The Hound of HeavenA Vision of BeautyThe Last ManWhite HeliotropeBelinda, and The Tune of Seven Towers.
(Which probably tells you more about my own tastes than anything else).

5) Read some author biographies if you want to create author characters.

I get suspicious when I start reading writer characters created by a writer. The main reason is the wish-fulfillment aspect that the stories usually take on. Either the writer character achieves some great success that the author of the story wishes she had, or the author uses the character as a stand-in to complain about the hardships of his life and How Editors Do Not Understand.
Oh, bull. Author characters don’t have to be caricatures or perfect Barbie and Ken dolls. Many of the classic writers I’m familiar with were deeply imperfect people. They’re still fascinating.
There’s Swinburne, who, to quote from the page I just linked, is far from the most perfect human being alive: “Once or twice he had fits, perhaps epileptic, in public; but he made this condition much worse by drinking past excess to unconsciousness,” and “He took a sardonic delight in what the critic and biographer, Cecil Lang, calls “Algernonic exaggeration”: When people began to talk scathingly about his homosexuality and other sexual proclivities, he circulated a story that he had engaged in pederasty and bestiality with a monkey—and then ate it.” There’s Dante Gabriel Rossetti, who was a painter as well as a poet—and stunningly talented at both—but suffered from drug addiction, mental breakdowns, the death of his first wife, poor health, and obsessive love affairs with various women which may or may not be described cordially. There’s William Sharp/Fiona MacLeod, one of a very few male poets to use a feminine pseudonym, who wound up marrying his long-time sweetheart—and then falling in love with someone else, and apparently not spending all his time angsting about it as a result, the way a fantasy protagonist might tend to.
There are also writers whose lives could provide a different kind of model, but who just don’t seem well-known enough, or are simply ignored. “Michael Field” is actually the penname of two women, Katharine Harris Bradley and Edith Emma Cooper, who were aunt and niece and also lovers. Many writer friends of theirs knew their secret; most of the people who read their poems probably didn’t. Natalie Barney, an expatriate living in France, was a writer herself, kept a salon in which other writers congregated, and had a long, happy, and flamboyant life that would serve well as a model in how to make a writer character who’s other than angst-driven. I’m sure you can think of other examples.
I’m tired of thinking that I can’t write about poets or fiction writers or non-fiction writers, and feeling wary when I read stories about them. There’s no need for that wariness, if authors will just a) separate themselves sufficiently from their characters and b) make those characters into real people. Reading some literary history and biographers should be the perfect cure for that.

6) What are literary communities like in your world?

It would depend on the cultural mood at the time, where in your world your writers lived, the level of technological advancement present, whether your world divided up genres and writing work in the same way we do, the way that the culture at large regarded it (at certain times and places in world history, poets have been feted, distrusted, regarded with awe, seen as threats to the government, and laughed at), and how writers of certain personalities and writing styles mingled.
Historical fantasies, urban fantasies, and science fiction are the only subgenres I’ve seen this concept addressed in (and in urban fantasies, of course, the author is generally using a literary community that already exists, not inventing a new one). Bards are the usual substitutes in other worlds, and their art is represented as music instead of writing. If the world is supposed to have a fairly high level of culture, that’s silly. There’s no reason that a world with complicated paintings and music and sculpture shouldn’t have an equally complicated level of poetry, prose, oratory, drama, and any other genre that you decide to add.
This is extremely useful for the cultural development of a fantasy world, even if you don’t write the whole story set in the literary community. You can get an idea of how people interact with each other in artistic professions, or professions in general, if you’re using this set of characters as a model for interactions in other sets. You can detail some of the art that usually remains vague “art” in the background of fantasy novels. You can have protagonists who aren’t fighters or mages (pretty rare). You can use themes of art and what it brings up, in addition to themes of war and saving the world and magic. And what happens when the writing starts interacting with the other aspects of the world, such as politics? Many, many real-world governments haven’t had the happiest set of arrangements with many writers.
That’s a lot more links than I usually give, but this is one of those rants where I can’t restrain myself from giving specific examples. I can see fantasy doing so much with these kinds of themes, without even breaking a sweat. I wish I could walk into a bookstore and find tons of books like this right now. Fantasy’s fantasy, sure, and a genre, and the little stepsister of science fiction, and the descendant of fairy tales and epics and myths, but it’s also writing, and there’s an awful lot to learn by reading what came before and taking inspiration from that.