Summers at Castle Auburn by Sharon Shinn (2001)
Little is mentioned about her time with her grandmother, focusing instead on the summers. Castle life is very different, with parties and feasts and all the luxuries of royalty. In addition to the regular servants, the family also keeps slaves from a magical people known as the aliora. Coriel has always admired her uncle Jaxon and hasn't questioned his prowess as a hunter of the aliora, and though she has known some of the slaves all her life it is only as a teenager that she begins to question the practice, and her admiration of Jaxon fades. Similarly, she has always been in love with Prince Bryan, but now that they are both older, she no longer envies Elisandra for being his betrothed.
This is not a story with a lot of action, at least not at first. The first several chapters are just setting the stage of Coriel's life and I felt quite drawn into her life and didn't care much that I had no idea yet what the story was going to be about. This allowed me to get pretty invested in her life, I think, so that later when all the things started happening I really felt like I was involved. Mostly it was a coming-of-age story with Coriel starting out innocent, but then learning more and really having her eyes opened to things going on around her, and how she deals with this new knowledge.
My only criticism is that I wish it would have dealt with the topic of the aliora a bit more. They are hunted and enslaved, but still don't seem to be considered "lesser" for reasons I can't mention without spoiling the story. Just a little bit more discussion about how people viewed them would have made it a lot more clear to me. But that's literally the only thing I can think of that was remotely wrong with the book, and even that didn't actually detract from my enjoyment, only made me wonder a bit.
I hadn't ever heard of this book until Jamie at The Perpetual Page-Turner posted a glowing review of it. Castles? Summer? I'm in! It is pretty much exactly what I expected/hoped it to be. I'm not a big fantasy reader* but here the fantasy elements were woven in with the larger story rather than dominating it. This is an older book so I wouldn't have found it on my own, and I'm very glad I heard about it. I read as much of it as possible while sitting outside on sunny days, and I highly recommend that you do too!
*I don't know what I think fantasy is. I have read and enjoyed plenty of fantasy (I mean, Harry Potter for one!) but when I think of it as a category I seem to think it's one I don't read.