You have two questions, the answer to one can change, and both will be only finally answered by talking to your manager.
- Is it possible to take sick leaves where I can be healthy myself but my wife/husband is sick? Would claiming the day off to attend to my
wife/husband qualify as sick leave or would I have to take a single
vacation because I am not sick myself?
For many companies, the answer to this is yes. You can take sick time when you are sick, when you need to go for a normal doctor visit (including eye doctor or dentist), for when children are sick or have doctor appointments, and when the spouse needs you to drive or accompany them to a doctor appointment. However, this is not universal, and if your HR manual does not spell this out, you should check with your manager before assuming it is true for this company. (My spouse has an annual eye doctor appointment in a city 4 hours away and has his eyes dialated, so I have to drive. I take that as sick. I also checked with my company first, because I would take it as vacation if they preferred.)
- Are there any issues with taking sick leaves during your notice period?
However, during notice periods, the standard company rules and conventions may change. Some companies pay out unused sick and/or vacation time, some do not. Some allow use of sick and/or vacation time during leave, some do not. IF your company allows you to use sick time for helping a sick spouse during normal times, and have been allowing it all along, they MAY allow you to use it (with the same frequency that you've used it in the past) during your notice period.
Since a notice period is used for handing off work, the general consensus is that you will be at work if at all possible. If you plan on spending all your sick time just so you can use it up, even without knowing your company, I can tell you that will not be looked upon well. A good recommendation from your company is worth more than four days pay.
The answer to the question in the title, how to ask your manager, is simply to go and ask. If they already know that your spouse is sick, and have let you take time for appointments already, another day should be allowed. Don't suddenly have the excuse of a sick spouse, and don't expect all four days. It is also valid to ask if unused sick time is paid out.
Boss, I have questions about leave during this notice period. You
know I've been taking my wife to chemo treatments every two weeks, and
another appointment will be next Tuesday. Is that still ok to take
even though my last day is the Friday after? Also, that will still
leave me with 3 1/2 days of sick time. Does the company pay that time
out, or is it just lost?
Boss, my wife just came down with Dragon Pox, and I'll need to take
the last four days of my notice period as sick. Sorry!