Note: This is the second installment in a blog series on startup security in a DevOps world (read the first here). This series is an adaptation of an e-book published in 2017, which was originally contributed to by JumpCloud CEO Rajat Bhargava and guest contributors Alan Shimel and Ben Tomhave. Read their bios below.
When working with technology, startups need to think about security concerns — both initially, when planning infrastructure and choosing solutions, and on an ongoing basis to proactively secure their applications and environment.
Because startups move quickly and need to remain agile, we recommend they work ongoing security into their existing workflows and cycles and look for security enablers and accelerators within their current tools and environment. The DevOps approach is one of the best ways to accomplish this, as it facilitates fast implementation, quick changes, and better success rates. This blog will discuss ways to approach ongoing security for applications, patch management, logging and monitoring, and incident management with a DevOps mindset.
The DevOps approach poses significant advantages for application development, deployment, and ongoing improvement, from more frequent code deployments to fewer failed change implementations and less outage-induced downtime. With this in mind, it’s important to take a DevOps approach with application security as well as development and deployment. Start by incorporating these three key lessons from DevOps that make application security more important and tangible to development teams:
1. You break it, you fix it!
In keeping with the culture of DevOps, resolution of security issues should be owned by the developers, not by security personnel. It’s one thing to allow a security expert to follow-up and ensure timely remediation, but culturally, it’s important that developers realize they own the responsibility.
2. Fail fast, learn faster!
Shortening feedback cycles is a critical element of DevOps, and this includes security. Application security testing (AST) should occur as early in the process as possible and feedback should be delivered directly to developers so as to more readily own and resolve any issues.
3. Mistakes are okay, but avoidance is also good!
Mistakes are going to happen. DevOps teaches us that we must allow for innovation and testing to occur. However, that doesn’t mean we shouldn’t first spend a little time thinking things through. A little forethought, especially around protection of data, can save a lot of pain later on.
Much can be said about application security, from teaching your developers to write better code to integrating AST tools into your CI/CD pipeline. All of these are good ideas and very worthwhile. Best of all, much of this can be done for reasonable costs. In fact, there are an increasing number of free and open source solutions for AST that will save you money. However, this also means you have little reason not to integrate these tools into your environment.
Additionally, it is wise to make use of pre-hardened images or containers, preferably with security tools already pre-installed. Developers should work off of these images to identify conflicts earlier in the process and ensure that builds will go smoothly.
Patch Management: Then and Now
A key area of emphasis for security architecture is planning for remediation and patch management. In a legacy world, patch-in-place is how things function. This can be frustrating, difficult, and fraught with risk. If you can’t get around the traditional patch-in-place approach, then you have to plan for it. Hopefully that planning includes how to get away from patch-in-place everywhere but your user endpoints (where we simply don’t have much choice today).
DevOps and the CI/CD pipeline provides us with an excellent opportunity to build for agility while also eliminating the legacy patch-in-place problem. At a minimum, we can automate the build and testing process such that you can update your host image or container, feed it into the pipeline, and out the other end comes your ready-to-deploy update or container, fully updated, happily churning away in the cloud.
Deploying in a blue/green manner can further reduce potential pain and suffering by allowing you to introduce new, patched images, test them in production, and then ramp them up while ramping the older images down, all without negatively impacting the customer.
Of course, to make this a reality, you must plan for it in advance. Ensure your product(s) account for session management in a manner that enables automated deployment. You’ll find that the serverless movement makes this planning imperative all that much stronger since you still need to be able to deploy updated containers or applications, and you will undoubtedly continue to have the same concerns around uptime, availability, and reducing the potential for negative impact to customers. Note here how engineering for (relatively) easy patch management ends up improving the overall resilience of your application and environment.
Logging and Monitoring
You do not need to go out and buy an expensive security information and event management (SIEM) solution simply to perform adequate logging and monitoring. Capturing logs is, overall, a fairly straightforward process. Implementation may vary a bit, such as deploying a Syslog server or leveraging S3 buckets in AWS, but the principles are still the same. Enable system and application logging for certain areas (e.g., various errors, login failures, traffic spikes) and then push them to a central location. There are many good tools on the market that will serve the needs of dev and ops and security. Don’t feel that you have to break the bank on buying a tool, just make sure you’re pushing everything centrally and are able to do some basic dashboards and alerts.
One practice area that is often overlooked until too late is preparing for incident management. Incidents are going to happen. It’s inevitable.
Break-fix scenarios abound in general, and that doesn’t begin to touch on various security concerns. In addition to having a reasonable logging and monitoring solution in place, you must also be ready to deal with incidents when they come along. Define your incident response process. Ensure you have contact information for all key personnel. And, perhaps most importantly, invest in some training around formalized incident management so that you can run your response efficiently and effectively.
Training programs will often spend time teaching how to set up a chain of command and reporting capability to deal with whatever crisis is facing your organization. Minutes, if not hours, of downtime can materially harm your business, and that doesn’t account for a potential reputation hit, which your startup simply may not survive. How you deal with a crisis speaks volumes about how reliable you are as a business.
For a more in-depth rundown of what to include in your security training, read our Security for Startups blog on securing employees and devices.
Make Strategic Security Choices
When it comes to ongoing security management, it’s important to stop chasing the next shiny object. Instead of adding tools to your infrastructure as they come along, choose tools strategically, so that a few robust and flexible ones can manage your needs collectively and adapt as you scale.
Look for opportunities to get extra value from security investments. For example, training programs for developers can often focus jointly on software quality and security. Cloud infrastructure investments and DevOps CI/CD builds can provide optimized frameworks into which security tools and practices are readily integrated. There are smart ways to leverage security to get wins in multiple columns. Seek those out. Don’t just view security as a cost center and an obstacle to doing business. In today’s modern era, strong security practices can be a sales enabler.
Further, the human side of security can’t afford to be ignored — especially in a startup, where work moves fast and employees wear multiple hats. In an environment where nearly 90% of breaches are caused by human error, startups need to commit to building employee awareness and training programs, setting clear security parameters and expectations, and cultivating a security-conscious company culture. To learn more about how to address the human side of security in your startup, read the next installment in this blog series, Security for Startups in a DevOps World: The Human Side of Security.
About the Authors
Note: Bios are as of the original e-book’s publication in 2017 and may not reflect current positions or work.
Rajat Bhargava, Co-Founder and CEO of JumpCloud
Rajat Bhargava is co-founder and CEO of JumpCloud, the first cloud directory platform. JumpCloud securely manages and connects employees’ identities to their systems, applications, files, and networks. An MIT graduate with over two decades of experience in industries including cloud, security, networking and IT, Rajat is an eight-time entrepreneur with five exits including two IPOs, three trade sales and three companies still private.
Alan Shimel, Founder and Editor-In-Chief of DevOps.com
An often-cited personality in the security and technology community and a sought-after speaker at industry and government events, Alan has helped build several successful technology companies by combining a strong business background with a deep knowledge of technology.
Ben Tomhave, Security Architect with New Context
Ben Tomhave is a Security Architect with New Context, a Lean Security company that automates the orchestration, governance, and protection of critical infrastructure and the industrial internet. He holds a MS in Engineering Management from The George Washington University and is a CISSP. He’s previously held positions with Gartner, AOL, Wells Fargo, ICSA Labs, LockPath, and Ernst & Young. He is former co-chair of the American Bar Association Information Security Committee, a senior member of ISSA, former board member for the Society of Information Risk Analysts, and former board member for the OWASP NoVA chapter. He is a published author and experienced public speaker, including engagements with the RSA Conference, MISTI, ISSA, Secure360, RVAsec, RMISC, DevOps Connect, as well as Gartner events.