The AP's David Germain has a write-up (with a way-clever headline) about Synecdoche, including a few quotes from Rock 'n' Roll, Hoffman and Jonze.

"The way I write is very much without kind of a goal," Kaufman said. "I have something I'm interested in and then I decide I'm going to explore it. I don't know where the characters are going to go, I don't know what the movie is going to do or what the screenplay is going to do. For me, that's the way to keep it alive."

"I tried to approach the directing in the same way. We have the script, we have the actors, and we're trying to figure out what this is, and you don't know what it is," he said. "You have to be open to what it's going to become rather than have this thing that you're trying to get to, which is boring."

... "People will learn to pronounce another word, and that's always good, right?" (Source)

IGN's Kaleem Aftab says Synecdoche "...is a zany, irreverent reverie that is at times brilliant, especially when setting up the fantasy world the characters inhabit, but by the end a tad infuriating and often incomprehensible."

Like all Kaufman scripts, the ideas behind the action are stunning; he's a master at setting up situations that always seem to run into cul-de-sacs and meet unsatisfactory ends.

...It's so ambitious and different that it has to be admired. Towards the end of the film it's stated that you have to go back to the beginning to understand what's happening and the film starts with a reference to British playwright Harold Pinter. As such Kaufman's film is best understood as a tribute to the memory plays with which Pinter made his name. But any greater conclusions will only come after we've seen it a few more times. (Source)

And Cinematical's review, in a nutshell:

The directorial debut of screenwriter Charlie Kaufman (Eternal Sunshine of the Spotless Mind, Adaptation), Synecdoche, New York is a sprawling, messy work of inspired brilliance and real humanity, a film that enthralls and affects even as it infuriates and confounds.

... Synecdoche, New York is the kind of movie that inspires more intellectual comparisons -- It's 8 ½ for our modern age! It's a post-Woody Woody Allen film! It's Jacob's Ladder for New Yorker subscribers! -- than emotional responses. (Source)

Thanks to Tim and Jon for those.