How to coordinate urgent video conferencing meetings with team when working remote with less than optimal Internet connection?

Question

I work as team lead of the IT Security team where I work. Today we had a security incident (verified via tripped canary tokens / observed IOCs) and I had to coordinate all hands on deck meeting with the team with little notice.

It has been my experience that incident response activities such as reviewing audit logs, reviewing system configurations, and restoring systems work best when team members are able to see and interact with each other, whether physically or via video enabled teleconferencing. Voice communication only just does not work well, as all participants being able to see what is being investigated (e.g: Audit logs or machine registry entries) is critical. Communication is also more easily distorted when the visual piece is missing.

However, several team members have less than optimal internet connection bandwidth at home, making video conferencing in real time difficult. For example, just today the connection often cut out. Due to Covid 19 pandemic, entire company is being required to work from home and I anticipate this to continue for the foreseeable near future.

During a fluid situation such as this, how can I better coordinate response when technology outside of the company control is constraining my work?

Answer 1

During a fluid situation such as this, how can I better coordinate response when technology outside of the company control is constraining my work?

Open a telephone conference call.

Phone lines tend to be far, far more reliable than what you seem to be experiencing with your internet connections.

If your real-time video is unreliable, send snapshots via email as needed. Again, email is more tolerant of weak connections.

Answer 2

Prepare ahead of time.

“On an emergency call” is not when you want to find out if people can connect. Set the expectation people need video-ready Internet and make sure the company is paying for whatever upgrade they need.

Practice. Have some “game days” where you exercise your security incident management process to work out kinks like this when there’s not a real crisis.

Use tools with graceful failover. Most civilized videoconferencing solutions you can also phone into if video fails. Share screenshots and logs via chat or other collaboration solution you have (this helps create an audit trail too, one would hope these things are being attached to a ticket or otherwise archived even if everyone’s video is great).

Evaluate your process. Does everyone need video going all the time? One screen-share takes a lot less bandwidth than 12 video streams. Do you really need all those people on a call, are they all really contributing to the incident? Or do you need a better incident management process and a stronger incident commander role who can update the interested onlookers while a core set of actual relevant people are working the problem?

Answer 3

If your team members have a poor internet connection due to lack of ISP offerings in their homes or lack of money to upgrade to a better plan/ISP, I don't see a way to make it work.

A workaround would be your company issuing mobile corporate phone numbers with a data plan, so the person could use it the mobile internet in an emergency with the company computer by their own cell phone or a USB dongle, but it would require a significant funding if a lot a people would require this (and it assumes that the cellphone ISP has coverage on each team member homes).

If the problem is due to a slow corporate VPN, you could ask your infrastructure team (if you aren't on it) to investigate and see possible fixes.

If the issue is specific to the tool you are using to do the calls (you didn't mentioned which one you are using), perhaps you could suggest another one for the company.