I recently had an interviewer ask this type question, so I explained how I looked at the rating system as a exponential system, rather than a linear scale.
The top of the scale of "10" are the devs who wrote the language, library, or technology. A "9" being someone with roughly absolute mastery of that technology, like someone with 20+ years of experience and has dealt with "everything".
An "8" would be someone who is at least a Senior position with lots of experience, probably a fair amount of formal training, and works on it on their own time. A "7" would be a mid to high level programmer that's dealt with a fairly wide range of problems over a handful of years.
A "6" would be getting into a Junior level dev, maybe right out of college that understand the quite a bit, but hasn't really done anything professionally.
Below that get into more basic understandings of parts of the languages, etc. Does the person understand the concepts of loops, collections, objects, APIs, etc.? A complete beginner (without any relevant or related knowledge) would actually be a 0 and should be able to progress to about a "3" in a year, with another 2-3 years getting to be a "5" or "6". Someone coming from a similar technology could get to be a "6" in a month and a "8" in a year or less, such as if they switched between Java and C# plus had an "8" level in the complimentary language.
So you can see that my view is completely different from what others see in the scale. I'd view myself as a "7" in my current role, but I've talked to some people who would think I'm a "9" and other's would think I'm a "5", due to the relative scale when comparing me to themselves.
Because this is so subjective, the scale definition varies so much, and you have the Dunning-Kruger effect (as mentioned in another Answer), you'll never get a good answer to this type of question.
You really do need to ask questions about what kind of things they've done before and how they solved problems. Then ask why they didn't choose a different method.
You: So, you solved this problem using an array, but why not an List?
Them: What's a List? {Definitely not the 7-8 they said they were.}
Them: Well, I didn't need all the extra methods and features that would have taken up extra memory and processing power. I just needed something simple to pull out pieces of data easily. {Probably the 7-8 they said they were.}
This is still subjective, but now it's you being subjective rather than the candidate. As long as you can control how things are evaluated, you'll have a better way of knowing what it is you really want to know. Sure, someone might be able build a Quick Sort from memory, but can they build a complex data structure and have the Quick Sort iterate over an ICollection of them?
Maybe you don't need an "8" and are willing to train a "6". This comes down to understanding your own needs and wants, and finding the right candidate for it as well as gearing your questions to those needs. A standardized test won't work for that. A "9" might breeze through it and consider it a waste of their time, while a "6" struggles and gives up. Same way with a standardized interview. If a "9" isn't challenged by the interview questions, they might not think it's worth getting hired, since they won't be challenged by the work. A "6" struggling to come up with answers during the interview might also give up there, due to thinking the job will be the same way. Not many people want to have a job where they are only utilized for 50% of their effort and knowledge. Nor do people want to have to spend 120% of their time, energy, and brainpower learning new stuff for the job.
So what I'm saying is that the interview needs to be geared towards the candidate, not just that you have a list of questions you ask everyone. You can have a list, but don't robotically ask every single question to every single person. It kind of sounds like you've got that already, so I'll stop here, since I think this is getting off track of your original question, anyway.