TLDR: I didn't know I was going to need a TLDR when I started. Using coaching early and as often as possible. Use it for praise and correction. Make feedback about problems not people. Set expectations and visit them often. Seek and accept feedback from them just as much. If feedback is regular, then it'll never be a surprise.
Coaching and feedback need to be given regular and often. They come in the forms of corrective suggestions, praise, instructions and thanks. You need to know when to use which, and how you use them will eliminate any potential surprises or offense.
Coaching is for minor things. Most things are minor.
Coaching comes generally in the form of questions, and sometimes in the form of a direct statement. When a mistake has happened, ask your direct questions about it. Don't be an interrogator. Do an examination of the process. "We achieved result {X} in this last sprint, and it fell short of our goal {Y}. What do you think led to that?" Don't directly make it about them unless they simply aren't getting to the thought space you need them to be in or refuse to accept responsibility for things.
Coaching is most effective when they arrive at the conclusion on their own. This self-genesis prevents any of it from being offensive because they can only be offended by themselves. Use nudging statements to get to the goal, and as they reach it, reinforce it by affirmatively acknowledging the conclusion and move directly to solutions. Try to keep the whole exercise about process, tools and circumstance. Don't let them (and don't let yourself) make the problem about people. You're dealing with a problem, and that problem shouldn't be about the person or anyone in they interact with. As someone once told me, separate the problem from the people.
Feedback is for the big stuff. Make sure it's actually big stuff.
Feedback generally comes in the form of statements and instructions. Personally I like to inform my folks that I have some feedback for them and then ask them if they're in an ok place to receive it. I do this exact same behavior when I have praise to give them so they don't associate this question with "Oh crap, I've screwed up and now I'm in trouble."
I still try to make it about questions and process as much as possible, and sometimes that just isn't going to cut it. Sometimes the feedback really needs to be "You aren't cutting the mustard, and I need you to be better."
The biggest obstacle to directs receiving feedback is blame. There are two ways I get away from this. The first is to accept the blame onto myself. This is especially useful if they're trying to blame a co-worker or someone in another department. You can immediately say "That's feedback I can take, and I'll see what I can do about it." Then switch the focus immediately back to the things that can be changed. Tell them very specifically that those are things that they can't control, so let's focus on the things that can be controlled - their actions, behaviors and responses.
Expectations are hard because you don't really expect what you say you expect. I know. That sounds absurd, and it really is how things operate. Often we'll tell a direct something like "I expect you to have those reports done by 4pm." The expectation we're actually communicating to them is that we expect their reports to be late. Only someone who is late with their reports needs to be told to have them by 4pm. If instead we were to say "When you finish your reports at 4, please send me a notification that you're done", we send a different message because there isn't a world where their reports aren't done by 4. So be very careful in how you frame your expectations. What expectations are you actually communicating? People will rise to the level of your expectations generally so make them high and communicate them effectively.
Feedback needs to be very regular, and it needs to go both ways.
I like to make my one-on-one sessions with my directs weekly. Not a lot of managers have the freedom to do this. Make them as often as you can without disrupting the work or the goals. Make this sessions their time as much as you possibly can. If they want to talk about the skiing coming up this weekend, so be it. Make their time a "safe space". That means (to me) they can say to you whatever is on their mind in any frame or context that they want. I even initiate all my new directs with this phrase: "This time is your time, and I need your complete unbridled honesty. If you're mad at me and need to tell me I'm being a douchebag, then say that exact thing to me." Don't restrict their speech or their feelings in these sessions. If you have feedback to give, make it plain. If they have feedback to give to you, accept it without rebuttal. Remember, their feedback will largely come from an emotional space so if you want them to accept your feedback you need to accept theirs (right or wrong). If the feedback you have has the potential to be especially harsh, say it. If you allow them to be completely honest, they will accept the same exact thing from you.
If someone says don't make it personal, they're a fool. It's always personal. To you it's business, to them you're telling them how much they suck. Try to make it about results. Try to make it about improvement of processes, techniques and behaviors. Don't make it about the past, make it about the growth moving forward. Make it a team effort between the two of you. "How can I help you achieve that?", "What things can I get out of your way?". Focus on "we" not "you". Their failures are your failures, so own every inch of them with them.
How can they be better if you aren't better? Always finish your sessions with some version of "How can I be better for you?" If you set the example of always self-improving through feedback, they will be more likely to give you constructive feedback. They will also be more willing to accept your feedback. "My boss knows she's not perfect, so it's ok that I'm not. She's trying to get better at what she does, so I should too." These sound like absurd statements to make out loud, and they still pass through our brains at light speed. We emulate the behavior we see in our managers most of the time. So if you want them to parse and accept feedback, seek it out for yourself. Most of the time they won't have anything, and when they do have something take it. Make it your own. Show progress on it, and report back to them with it. Be the example of what you want them to be.