How to teach and document intuition and experience?

Question

A part of my role as a technical team lead in cybersecurity team at my workplace is to mentor more junior security engineers. While other job roles may have hard rules to follow that almost always apply, such is not the case in IT security. We have best practices but implementation and analysis of adequacy requires intuition and experience that only comes with work experience. I have been working in cybersecurity for slightly more than 6 years and they about 2 years.

As an example, when performing pen tests, reviewing pen test findings or granting security sign off on a user story, there really isn't a set of hard rules that if met signifies one of our products is secure or not met, insecure. One must understand risk management, threat landscape, and also draw from past breaches. How such factors weigh will also be different from product to product.

In my guidance documentation, I don't want to handhold junior team members by being excessively prescriptive of what they should or should not be reviewing in each use case as I believe such approach hurts employee development and because IT security is a very dynamic field.

However, junior team members have given me feedback that they find my mentorship and guidance too theoretical without enough concrete details. I have been getting increasing requests from them to list very specific , prescriptive points to review in scans of products, something I am very reluctant to do.

How do I help more junior team members improve their intuition and "feel" for IT security without spoon-feeding them every fine detail?

Answer 1

How do I help more junior team members improve their intuition and "feel" for IT security without spoon-feeding them every fine detail?

Do the tasks together and coach by asking open questions rather than telling them what decision to make.

As an example, when performing pen tests, reviewing pen test findings or granting security sign off on a user story, there really isn't a set of hard rules that if met signifies one of our products is secure or not met, insecure. One must understand risk management, threat landscape, and also draw from past breaches. How such factors weigh will also be different from product to product.

What works for me is to teach such an "expert decision" tasks is to do the review together and let the mentee do all the work, with me in the background asking questions.

• "What is your impression of this pen test?"
• "What threats are being is discussed?"
• "How does the tested product deal with scenario X?"
• "What do you think about these results?"
• "What is your final verdict then?"

Ask the questions you would ask yourself, but don't answer them yourselves. Only step in, when the mentee draws the wrong conclusion, or even better let them figure out by themselves by asking more leading questions.

When it comes to conclusions be open minded and accept verdicts you wouldn't have necessarily found yourself. E.g. if you have to decide whether to approve or reject a test report and would go for "reject", but it is a close enough call that "accept" would be reasonable too, then don't correct your mentee if they go for "accept", as long as their reasoning is substantiated and they not missing anything.

If you keep doing these discussions, you will find some things you will have to explain over and over again. These are the items you would want to capture in a written documentation, to save you some time.

Answer 2

Wikipedia defines intuition as "the ability to acquire knowledge without recourse to conscious reasoning."

Professionals should always be looking to convert their sub-conscious reasoning, into conscious reasoning, evaluate and understand the reasoning, and document it for others.

Structural engineers don't rely on intuition. When looking at building plans, there are models they use to determine if a building is safe. If their intuition leads them to find something that wouldn't be found, they update their models to ensure its checked in the future.

Pilots don't rely on intuition. When there is an emergency, they go through a checklist. If the checklist doesn't pick something up, the airplane manufacturer will update the checklist.

This doesn't mean that intuition serves no purpose. Intuition may help you find things quicker, and more efficiently. It also you finds gaps in your documentation.

One must understand risk management, threat landscape, and also draw from past breaches. How such factors weigh will also be different from product to product.

Why is it not possible to document these things so that CAN be understood? You're relying on a lot of implied knowledge, which is impossible for them to know. So ensure the knowledge is not implied, but documented. This also helps ensure it is correct.

It also helps validate the modelling that your more senior members do.

Answer 3

Training is best done in detail with a manual for reference. You cannot teach intuition, and it is not how cyber security works. Your method is not intuition, it's years of details being put together, nothing esoteric about it, most professional fields are the same.

You need to go to basics and do the details and work your way up. If products differ you go through the whole process again specific to the product.

When details change, you update the manual.

If it's not done like this the company risks multiple potential points of failure. The idea behind proper manuals is that any suitably qualified person can follow instructions and familiarise themselves with everything while performing the necessary tasks.

Answer 4

How about Case Studies?

You walk them through every step of your thinking and testing for a given scenario. What factors you weighed, decisions you made, and your eventual analysis.

This allows you to give them the details they’re asking for while also (hopefully) making it clear that there is no one size fits all approach to what you’re doing.

Added bonus:

Write down 2 or 3 of these and you’ll probably see a lot of repeated steps. Congratulations! You’ve just found some concrete details and considerations that your juniors can look out for in their future reviews.

Answer 5

In my guidance documentation, I don't want to handhold junior team members by being excessively prescriptive of what they should or should not be reviewing in each use case as I believe such approach hurts employee development and because IT security is a very dynamic field.

Don't write the guidance documentation all by yourself. Do it on a wiki. Assign your juniors a couple of sub-topics. Do a couple of sub-topics yourself first to set the tone and the overall structure.

If they've overlooked something, you can fill in the gaps. And if you've overlooked something, maybe they can help fill in the gaps also. The point being. If you put them in charge of trying to teach others a particular topic, there is no better way to learn it themself. See the Richard Feynman learning technique.

Do the same if you're having a meeting about this. Instead of having a single person present to others and have everyone else remain passive. Have a roundtable discussion and have one or two people scribe, or have everyone prepare and present a subtopic for 2 to 5 minutes

Answer 6

I have been getting increasing requests from them to list very specific , prescriptive points to review in scans of products, something I am very reluctant to do.

How do I help more junior team members improve their intuition and "feel" for IT security without spoon-feeding them every fine detail?

Alternate between spoon-fed/specific details and your current, more theoretical guidance.

If they feel you are too theoretical, then try now and then dropping some specific insight or points to review in scans (i.e., showcasing a bit your know-how, intuition, and experience).

Usually, at first you may want to "spoon-feed" a bit more, and then as they progress the spoon-feeding will be needed less and less, as they develop their own intuition/experience.

Answer 7

Let's restructure the question into the two key questions:

1. How do we teach intuition or intuitive thinking?
2. How do we document experience?

Both are very doable but at different levels of detail.

Intuition is a macro or bigger picture thing which we can really sum up as culture. The need for I.T. to create a culture around security and safety needs to be at the forefront of the company culture. We can teach this through behavior modification techniques:

• Create easy to remember advice / guidelines (e.g. "Too good to be true = it is", "Slow down and think before acting", "Question everything, do not blindly trust anyone", "Ask the I.T. help desk or a manager via a known phone number or email", "Don't click links without carefully inspecting them", "Don't give information out willingly especially personal or private information. When in doubt, it's private and a secret", etc.)
• Create routine internal tests (e.g. Send phishing email tests, Do regular audits, explore physical device security [people leaving devices unlocked], etc.)
• Require routine and continuous training (e.g. self-paced training on I.T. threats, continuous simulations of various incident types and/or events, external testing [hire firms dedicated to testing their ability to breach your security ... keep it on a need to know basis] and auditing, automatic (re)training on people who fail internal I.T. tests, etc.)
• Send routine reminders and examples of "Do not do..." or "Do the following..." which are "tips & tricks" style company wide memos/emails
• Remove individuals from sensitive positions who exhibit problems with maintaining I.T. standards and security
• Conveying a spirit of "You ARE the first line of defense to stopping problems and disasters before they come crisis and failures" with EVERY employee from the CEO to the janitor is of the utmost importance ... but when something does go wrong and that first line of defense fails: having excellent backup systems and the sense "Contact I.T. immediately as time is of the essence in containing and correcting the problem(s)."
• Cultivate a thirst within the team for constant improvement and research of the latest technology trends, cybersecurity threats, and best solutions/strategies for mitigating those threats (e.g. reading credible articles/books/blogs/etc., visiting cybersecurity oriented websites/forums/conferences/etc., and diving into the I.T. world from all angles [esp. ethical hacking ... learning how it's done to prevent it happening in your company]).

Documenting experience is a micro level concern where individual incidents or cases need to be logged in a documentation system (e.g. knowledge base, FAQ, and/or Wiki) that is a S.P.O.T (Single Point Of Truth). Whatever system is used needs to be easy to use both from a "file it" perspective and a "retrieve it" perspective. Good search tools, categorization, and organization are essential to providing a solid documentation system that will be used, maintained, and updated.

Whenever there is a failure or incident, the entire incident must be logged in the system. Everything should be covered from a "Step-by-step breakdown of what happened by all actors" to "What did we do right" / "What did we do wrong" and "How did we fix the problem/enhance the security of the company" point of view. There should be an investigation in all cases that ultimately leads to this documentation getting created and appropriate new policies, procedures, and practices being put into effect immediately.

Establishing standard operating procedures (e.g. checklists and policies) for disasters, breaches, and other I.T. critical events (incl. termination of employees/contractors/etc., transferring users, absorbing new business units/companies, etc.) should also be put in this system (with great detail). All checklists and associated reference materials (e.g. system manuals, accounts, shared credentials, servers, etc.) should be placed here as well. A good book on checklists is "The Checklist Manifesto".

Creating routine processes that update and ensure the information in the system is accurate, current, and clear is just as essential as well (doing this around the holidays when there is a low volume of work is a great idea). Also, creating routine audits of the system are essential to test it functions as intended and is improved in the areas where it is lacking (e.g. information is not easily found or categorized, an incident is not logged, the correct procedures is no used, the system is not consulted for an incident, there are enhancements needed for specific cases). A knowledge base system is only as good as it is current, clear, and correct.

This is the kind of approach used (and even enhanced) in the most successful companies and I.T. focused teams/organizations. Regularly consulting external resources such as Microsoft best practices/feedback, Deloitte/I.T. professional consulting firms, and government agencies feedback/regulations is always a great step ensuring you, your team, your I.T. department, and your company are staying ahead of the latest threats and risks in the I.T. realm.

Best of luck! :)