how do HR departments filter applications when they don't have expertise in the field, and years of experience is no sign of competence?
Badly, but they have no better measure.
As you note, there is really no good way, so they choose what they think is the quickest and relatively efficient way to gauge the expertise level of candidates (of which there can be lots so speed is usually more important than perfection). They do acronym pattern matching and filter for years of experience as a loose measure of expertise. I think most of them know that there is a weak correlation only, still it is better than no correlation at all. However, their job is not to find the most suitable candidates, only to weed out most of the inadequate ones.
(And they aren't expected to be perfect even with that; if they passed through some crap candidates, they can almost always explain it with something like "sorry, these are the best we could get - maybe your job description is not enticing enough, and anyway we know that our company is not Google" * - on the other hand, if they filtered out a perfect candidate, noone will ever know.)
Practice makes the master?
Another - potentially unconscious - cause for this may be the widely known idea of "ten thousand hours of practice makes a master". This translates to about 10 years of professional experience. Of course, the problem with applying this principle at face value is, repeating the same one year of basic experience ten times does not make anyone a master. One has to regularly get out of his/her comfort zone, explore new territories, get and solve new challenges in order to get into and then stay in the top league.
A complementing criteria
So another useful signal (which in my experience HR is checking) is average length of employment relationships. Let's say HR got CVs from three developers who spent the same length of years on the field, but
- A stayed at the same company, working on the same product in the same team all the time,
- B worked at 4 different companies over the years,
- C worked at 12 different companies over the years.
Chances are A has the least experience. B probably has significantly more experience, moreover (s)he managed to get a job already at 4 different companies, and also to keep it for several years, so she is most likely good at what she is doing. C, from his part, managed to get employed at even more companies, however his/her jobs lasted for less than a year on average so (s)he is most likely seen as a risky hire. Thus HR will most likely pick B as the best candidate. And if A and/or C get through the filter, they will get some tougher questions to prove their experience and their social abilities, respectively.
* except when it is Google of course. But Google HR probably needs no such excuses.