Security Engineer Checklist | Datadog

The Security Engineer’s First 90 Days checklist

Being one of the first security engineers can be quite daunting. This security checklist aims to help security engineers and CISOs in early-stage companies to prioritize their efforts in the 3 months of their new job.

Timeline

  • Process

    • Don't just take your previous experiences at past companies as the go-to model for your new company. While some things will be the same, several aspects will undoubtedly be different, and doing exactly what you’ve done before won’t be as effective. If you’re coming from a more mature company, it's easy to suffocate an agile startup with heavy security that does not scale well, for example. Security engineers operate inside a business, and understanding that business before enforcing GovAgency-like security measures is key.

    • Even though it may not have been somebody’s specific role, someone was handling security aspects for the company before you started. Take the time to meet early on with the "security champion", not only to gather precious information about the current state of things, but also to agree on his/her scope onwards should the person want to stay involved in security tasks. Depending on the company, this person might be the CTO or a security-minded developer.

    • As part of your first week exploration, you need to gather enough information from key stakeholders in order to have a clear understanding of the product development processes (e.g. steps, key milestones, teams involved, governance structure, etc.). These might be catalogued in documentation or through detailed oral explanations that should be written down. Deeply understanding the product development processes and structure of the engineering organization in your company will serve as a basis when you start to introduce security awareness and tasks within the product development lifecycle.

      Learn more

    • Third-party providers and agencies need to be managed from onboarding to offboarding. This entails a thorough due diligence before and during the relationship as well as frequent risk assessments to keep abreast of the level of access the provider has and the potential vulnerabilities involved. The contract termination step is often overlooked and should be well prepared for during contract drafting, notably in terms of data migration and access removal. A checklist of all the tasks to be performed during onboarding and offboarding should be set up and regularly updated.

      Learn more

    • Onboarding and offboarding are important security moments for your company. You’ll want to ensure that new employees enact the security measures needed and that your company follows the appropriate steps for employees who are leaving.

      Your onboarding checklist should contain a list of all the steps you and they need to follow when an employee, contractor, or intern joins your company. A similar list can also be used when someone is leaving. Ensure that you deprovision all accounts they had access to during the offboarding.

      Learn more

    • Work with your developers to set up a process and a checklist for security code reviews in order to empower them to run manual and automated security code reviews themselves. Be available to answer their questions and be ready to assist if needed.

      Learn more

    • You should add some checkpoints to the SDLC so the developer teams will think of the security team when they’re creating and updating applications. Some tools or checklists allow for asking developers a small set of questions when starting a project in order to let the security team know how much attention they should pay here. For instance, "is this service gonna be exposed on the Internet", or "Is this service handling customer data" are questions that can help you quickly get a sense of how involved you need to be on any particular project.

      Learn more

    • Define what constitutes a security incident and design the response plan outlining the necessary tasks and roles. Share the response plan widely and make sure the employees are aware of their roles through regular training and simulation exercises. You don’t want to be in the position of needing to wing it during the crucial time after a breach or major incident.

      Learn more

    • With the amount of tasks required as the first security engineer, you can easily drown under less important tasks, resulting in losing track of serious unresolved vulnerabilities and substantially diminishing your incident response capabilities. Automate as much as possible in order to free up valuable time for tasks that actually require human expertise and deeper analyses. Take advantage of the multiple solutions offered in the market and of computational analytical power.

      Learn more

    • Before embarking on independent security assessments and penetration tests, it is good practice to run checks and correct some commonly identified issues (such as missing patches, weak or default passwords used, unsupported operating systems or missing input/output data validation) in order to use the external auditors time and expertise on more subtle issues.

      Learn more

    • As a security engineer, you might also be the go-to resource for sales teams that require help filling in security forms. Spend some time retrieving and structuring all the previous requests to save time for future questionnaires.

      Learn more

    • Draft security policies and procedures for the company, and, more importantly, communicate and circulate them. Make sure they are easily accessible and make them understandable for your co-workers. Set up a process to review and update them regularly at a certain frequency or when a specific event occurs.

      Learn more

  • Culture

    • If it was not included in your onboarding documentation, ask for a list of the key stakeholders in your organization, be they developers, ops, execs, or managers. Your manager might see the importance of accompanying you to introduce you. Arrange together to meet with them and discuss their understanding of security, your role, and their concerns.

    • As a general rule of thumb, adopting a humble and respectful demeanor is a factor of success for every newcomer within an organization. It's tempting to show off how much you know about security and cyber-splain to everyone how insecure their setup is, if only out of a desire to establish your authority and credentials. But this will backfire on you. Being too hasty and judgmental in pointing out the shortcomings in the company’s security will not earn you the respect of your new colleagues, rather it will drive them away. Take comfort in the fact that if the company deemed there were no issues, you would not have been hired!

    • Liaise with your people team or your CTO to set up a targeted security training for all employees, whether they’re engineers or not. The training should not be just a list of instructions, but rather a clear explanation as to why certain rules have to be put in place. You can include technical details if necessary, but make them accessible for all skill levels. The training should be included in the onboarding process.

      Getting buy-in on the security policies you’re implementing will make them much more effective, and a good, empathetic training will help you do that.

      Learn more

    • The security community is generally a friendly one, and you can learn a lot from other security-minded professionals. As such, if you are not already a member of a professional group in your area, look for security meetups and communities, both online and offline. You could also reach out directly to fellow security engineers, whether in the same business line as you or not, to exchange ideas about your jobs and responsibilities or to discuss how they navigated being one of the first security engineers in their organization, if they were.

      Learn more

    • Managing security is an ever-changing landscape, so you need to keep yourself updated on the practices, tools, zero-day vulnerabilities, patches etc. It can seem overwhelming, but there are many websites and newsletters in which you can get regular information.

      Learn more

    • Don’t make security a one-day annual training everyone has to go through and then forgets about. Permanent and contract employees need to be aware at all times of security threats, beginning with how they set and handle their passwords, use their emails, and secure their laptops and external drives.

      Learn more

    • You have to stay on top of emerging vulnerabilities to effectively protect your systems and data. New vulnerabilities are constantly discovered and threat actors continuously seek ways to exploit them. Follow the key industry threats to be proactive about this.

      Learn more

  • Vulnerability Management

    • If your company has an issue tracking system (such as JIRA), make sure that security-related issues can be identified easily or work with the team managing the system to create a special flag or project. Communicate this new category to your co-workers and clarify how and when to use it. Compile all the issues you uncovered during the general and specific security audits. If no security issues are in the backlog… there is probably a cultural flag here!

      This will make addressing and prioritizing security issues much easier for you.

      Learn more

    • Oftentimes, even if vulnerabilities have been reported (in JIRA for example), they have not been addressed prior to your arrival because people did not know they had to address them, did not realize that they needed to be fixed immediately, or did not have or allocate the resources to assess and fix the issues. Getting a handle on the backlog of reported security issues and prioritizing fixes is a good place to start after you understand the systems your company has in place.

      Learn more

    • Do not be alarmed or overwhelmed by the number of vulnerabilities uncovered during your audits. All of them do not need to be fixed right away; you can draw up a plan to fix them over time. However, do not defer fixing the most critical issues. If you identify a serious vulnerability during one of the audits and security reviews, you should pause and fix the issue immediately. If you can’t fix it, mitigate it.

      Learn more

    • A bug bounty program will allow external hackers to report vulnerabilities. Most of the bug bounties programs allow you to offer rewards for bugs found. A lot of the reports won’t be valuable and you need security-aware people inside your development teams to evaluate the bugs you receive. These programs are good additions to other security initiatives as it incentivizes people outside your company to share bugs you may have overlooked.

      Learn more

    • Once you have a baseline audit of your company’s security across the major areas, you should schedule deeper tests and assessments in all areas (infrastructure, applications, people). These will give you a complete picture and the baseline you need to make strategic security decisions.

  • Cloud Security

    • Review the elements of your cloud architecture and the interfaces between them. By having a look at the observability solutions leveraged by your engineering teams, you should be able to map your cloud accounts and run automated tooling to discover all storage buckets, databases, and compute resources. Your cloud infrastructure will evolve continuously so bonus points if you’re able to do this continuously and get alerted when new cloud assets are spun up.

      Learn more

    • A large number of security breaches are still caused by misconfigurations. There are many layers of configuration options that could create exposure, whether it’s on the identity and access management policies, network configurations or storage policies. Mistakes are easy to make. You should be able to identify misconfigurations exposing you to risk and address the relevant ones accordingly. Cloud Security Posture Management products typically help with this.

      Learn more

    • In today’s business world, company data is the most precious asset and backups are therefore crucial. Check the integrity of previous backups and make sure the settings are correct for future backups with sufficient storage space and backup frequency. If there are no backups, set them up immediately.

    • If you have been handed a list of the exposed services, make sure it is up-to-date or take time to update the exposed machine’s information first and schedule to update the list thoroughly as soon as possible. One good way to create this list is to use cloud API (AWS to list Route 53 domain names, or EC2 instances) or to leverage a Cloud Security Posture Management product.

      Learn more

    • As more and more day-to-day business activities and revenue rely heavily on access and interactions with your website, it is important to audit your DNS as soon as possible and regularly afterward. Without proper security on your DNS, attackers could extract a list of all your assets or steal your domain names, which means they could lead efficient phishing attacks on your customers.

      Learn more

    • Logs are a valuable asset for getting signals from your production environment and for investigating suspicious activity or a security breach. A centralized log platform helps you to make the most out of the analytics potential held in your logs and provides a view across all themes (applications, network, users, etc.)

      Learn more

    • Secrets, such as private keys, are extremely sensitive data and must not be stored unprotected. They should be securely stored in a vault. Some vaults can manage certificates as well.
      Secrets committed in an application’s source code should also be tracked down and stored elsewhere.

      Learn more

    • When IAM roles are granted excessive permissions, individuals may have access to resources and actions beyond what is necessary. Attackers could leverage compromised accounts with elevated privileges to move laterally within the infrastructure, escalate their access, or exfiltrate sensitive information. Regularly auditing and reducing the permissions of over provisioned IAM roles is crucial to ensure that access controls align with the principle of least privilege.

      Learn more

    • DoS and DDoS attacks can be devastating for a business’ bottom line. Depending on your business, disruption to the availability of your service could make you lose out on revenue or impact your customers. Taking actions to protect your systems and mitigate the effects of these types of attacks is key.

      Learn more

    • Create a security dashboard to give you an overview of the security efforts, or implement a tool that will do it for you. Avoid manual reporting -- all data should be automatically provided by the solutions you use.

      Learn more

    • Make sure to follow the latest security releases and update your infrastructure as soon as they become available. Having servers without public IP addresses (e.g. in a VPC) and rate limiting authentication services are some of the measures that can be implemented to protect your servers, and consequently your applications.

      Learn more

  • Application Security

    • You cannot secure what you’re not aware of. With web applications and APIs being one of the main entry points for attackers, it’s crucial to understand your landscape and the key engineering owners you’ll need to team up with. Usually the observability products used by engineers for reliability and performance are a good starting point.

      Learn more

    • Application security is increasingly one of the top security concerns for modern companies. On early audit you’ll need to do it on your applications. You’ll want to gather answers to questions like:

      • Are your applications using vulnerable or outdated dependencies?
      • Are they accessing the database?
      • Are they handling authentication?
      • Do they rely on a framework (Rails, Symfony) or are they using in-house components?

      Learn more

    • When security researchers discover vulnerabilities in the web services of your company, they will need a way to report them properly to you. By adding a security policy, such as security.txt, to your website, you help them easily get in touch with you about any security issues they uncover. You should mention that you support responsible disclosure, allowing you time to assess and fix the reported vulnerabilities.

      Not having a clear means to communicate security vulnerabilities will either mean that they won’t get reported, or that they will get lost in the shuffle of your company’s generic "contact us" inbox.

      Learn more

    • Infuse security into all steps of the product development process and not just at the testing phase. Security-minded developers should check the dependencies in your applications for known bugs and vulnerabilities before using them and keep them updated when zero-days are found or patches are available.

      Learn more

    • Be available to your engineering team for security reviews of the architecture and update the architecture documentation regularly. You want your engineering team to come to you and work closely with you on the entire gamut of security needs, so be as present and helpful as possible.

      Learn more

    • If your security practices impact your development velocity, they will be looked at as more of a burden than a value. The best practices today are to take lessons from DevOps and find ways to bring security closer to developers. Leverage tools that can automate security checks and monitoring. Implementing automated SAST/DAST tools, vulnerability dependency scanning, and others will help you catch the obvious flaws before they get into production. Just beware that you’ll have to sift through false positives and that these tools will not be able to identify flaws in your business logic or once your tools are in production.

      Learn more

    • Doing as much as you can to catch security vulnerabilities pre-production is helpful, but without the full context of runtime, you won’t be able to catch everything. Protecting and monitoring your applications in production, in real time, can greatly improve your security posture. Tools like WAF and ASM products can give you visibility into the security of your production applications and help you stay on top of what’s going on.

      Learn more

    • The attacks representing the most significant business risk for organizations are often attacks targeting sensitive business functions of applications rather than common vulnerabilities. Work with business and engineering teams to identify the biggest threats and implement monitoring and protection solutions to automatically remediate these threats.

      Learn more