You need to weigh up the risk of your background check throwing up problems and what will happen if it does.
- It may come back clear: no problem -- you can carry on working.
- It may show some minor problem which means you get moved on to other things.
- There may be a major problem which means that your employment offer is summarily withdrawn and you're fired. You won't have any comeback in this case because you have accepted the job is contingent on the background check.
Only you know whether there is anything in your background which could pose a risk. It may be that the company is trying to save the cost of the background check — if you refuse the conditional offer, they have saved the cost and managed not to hire someone with a potentially dodgy background.
I do know of one case where a background check was failed because the company received details of someone with the same name and date of birth as the applicant, but it was a different person. That was sorted out, but it took some time. If this happens to you, you will be sacked and have to prove your innocence.
For myself, I don't think I'd take the chance. If they want you, they will wait for you, and satisfy themselves in the meantime. And if something happens that they can no longer take you on, then at least you still have a job. But only you can analyse the risk.