As for me, he just tried to reject an attending the conference trying to joke. Sense of humor is different form a person to a person. Obviously your company has no budget for an attending conference. Saying you - we can't afford it, can alert you, because you will understand that company doesn't doing well. And as result you will start looking around. Certainly you are very valuable employee for the company, so he decided to do the trick. Another confirmation of my guess, why the company didn't send you to the conference alone? They could save on attendance fee for your coworker. Your manager just clumsy joked instead of a straight reject. When you work remotely you do not communicate with people much beyond of your job duties and as result personality of many people is hidden for you, and as result you can't explain certain their behavior.

As for me, he just tried to reject an attending the conference, trying to joke. The sense of humor is different from person to person. Obviously, your company has no budget for an attending conference. Saying you - we can't afford it, can alert you, because you will understand that the company isn't doing well. 

And as result you will start looking around. Certainly, you are a very valuable employee for the company, so he decided to do the trick. Another confirmation of my guess: Why didn't the company send you to the conference alone? They could save on attendance fee for your coworker. Your manager just clumsy joked instead of making a straight reject. When you work remotely you do not communicate with people much beyond of your job duties and as result the personality of many people is hidden for you, and as result you can't for certain explain their behavior.