I want my manager to know my actual background and areas of expertise and feel guilty that I “stole” a position using someone else's credentials. However, I'd also like to retain my position, if possible. How can I tell my manager that I was hired by mistake?
Skip the "hired by mistake part"mistake" part - that's just silly. Deal with the "my manager doesn't know my background part"background" part. You don't really know if your manager was confused during the hiring process, or is just now remembering details incorrectly. Deal with the real issue, not the imagined one.
Talk to your manager in private. Something like "You know, you mentioned [wrong previous employer]. But I never worked there." should start the conversation.
Don't make it about "You hired the wrong person." Make it about "You are confused about my background".
I want my manager to know my actual background and areas of expertise
That's reasonable.
and feel guilty that I “stole” a position using someone else's credentials.
Unless you did something intentionally, that's nonsense. You didn't steal his credentials any more than you stole his identity.