> ask why the author did certain things instead of certain other things,
I think that the answer to that is unknowable to any mind other than the author's, and the best speculation can come from someone who's digested many of their work for a long period of time. It's certainly an interesting discussion to have if you can find enough people like that but it's tough!
You can definitely make a good guess at it in some cases. It’s often easier when the writer’s not very good at e.g. plot.
Getting too good at it usually means you start automatically spoiling a lot of books for yourself, though. “Why’d this character show up in chapter 3? What do they do for the story? Ohhhh they’re secretly the villain” or “why are we spending so much time with this character right now, in this way? Oh, the author’s gonna kill them.”
I guess you're thinking more plot oriented narratives.
I was thinking about something like Ulysses, Virginia Woolf, Proust, where the artistic choices are more stylistic and about examining thought itself, rather than structurally about characters and events.
Ah, sure, that does work differently. You’re not so likely to spot the seams of the author’s stitching-together of the plot in those. (Any readers who’ve wondered, unprofitably, what literature nerds mean when they separate out capital-L “Literature” as a distinct category: that right there is one factor that tends to push it into literature territory)
Though if one is not constantly asking “what might be intended here? What might this signify?” and similar questions while reading those kinds of works, I’m not sure what one is doing while reading them. That’s not identical to asking what the author was thinking, but I do believe that considering why the author chose one thing instead of another (why doesn’t the boat get to the lighthouse? What would change if it did?) is often worth doing, and I think sometimes one can arrive at something like the author’s reasons (even if it can’t be known for sure).
[edit] “But who cares if some reading of an episode or element of the story is the one intended by the author?” is the obvious question, then: I’d say if we can get at a plausible intended reading, with some reason and evidence behind it, that’s likely to present a more-useful lens or jumping-off point for unraveling and drawing meaning from other parts of the work, or even the author’s other works, than any given WAG one might think up. It’s not necessary to create and support a hypothesis of what the author had in mind at every turn, but if likely intent can be teased out, it helps direct profitable lines of inquiry elsewhere.
I think that the answer to that is unknowable to any mind other than the author's, and the best speculation can come from someone who's digested many of their work for a long period of time. It's certainly an interesting discussion to have if you can find enough people like that but it's tough!