I've got a formal job offer, conditional on the background checks.
That is good, but that doesn't mean that you have something that is 100% guaranteed. The offer can still be rescinded, and the start date is unknown.
Because of my sector, the background checks are being detailed and
intrusive.
That makes it very hard to know how long this will take. I have seen detailed background checks take days, weeks, months, or even longer.
The prospective new employer wants to contact my current employer as
part of these background checks.
You current employer is probably the worst reference to call. Even if they will answer the background investigators questions, and most won't, they will tweak their responses to help their situation. If you are a great employee they are unlikely to give you a glowing reference. If you are a poor performer, they may hype your performance to get you out of their hair.
Furthermore, the recruiter even wants me to give my resignation and
notice as soon as possible to my current employer so that he can
inform the hiring manager about my start date.
Never give notice until after everybody has signed the non-conditional paperwork. As long as there are conditions, the company can still decide to walk away. But you can also walk away from the offer if it is taking too long for those conditions to be removed.
Don't resign if they need to to a background cheek. Also don't resign if they are waiting for their customer to approve, or for them to win a contract. Watch out for the "waiting for the budget to be approved", that can take months.
I am not comfortable with this situation and can see very clearly how
this can backfire. Since these background checks are being so rigorous
and taking a long time, it is crystal clear that the probability of
failing the background checks is not zero.
Background checks aren't an absolute standard. Two companies can read the same report and one could decide the one issue is a red flag, and the other might not be concerned.
I recently had a relative that was told the drug test and background check would take less than two weeks. A month after the drug test was done, the background check company admitted they admitted they had no record of the drug test, and therefore hadn't even started. The entire process with holidays and other delays meant they started almost 6 weeks late. good thing they didn't resign until everything was addressed.
In that scenario, I risk losing both the prospective and the current
job, burning bridges with my current company.
How should I proceed?
Don't resign. Discuss what you are comfortable with regarding the background check process. Discuss your required notice period. Then set a course of action that will happen after the last condition is removed.
Many companies in the US outsource the proof of employment to a 3rd party. They provide a process for confirming dates of employment, and very little else. This is done for both current and former employees. They also provide pay level if the check is being done for a loan.
Unless your current company is small they will have a hard time contacting your current supervisor unless you provided name or contact information.
If the new company isn't conformable with your requirements keep looking. In fact even if they are comfortable with your requirements keep looking. You should continue to search, apply, and interview right up until the start date.