In this multi-part blog series, I will describe some microservice security design patterns to implement micro-segmentation and deep inspection in the interior of your Kubernetes cluster to further secure your microservice applications, not just the cluster. I will also demonstrate the design patterns with working Proof of Concept deployments that you can use as a starting point.
Follow up posts:
– Part 2 Setting up the Insecure Deployment
– Part 3 The Security Service Layer Pattern
– Part 4 The Security Sidecar Pattern
– Part 5 The Service Mesh Sidecar-on-Sidecar Pattern
There are many tutorials on microservice security and how to secure your Kubernetes clusters and they include many of the following topics:
- Checking for common cluster configuration errors
- Configuring encryption for cluster components
- Configuring access controls to the cluster components
- Using trusted container image repositories
- Checking for common security issues in code during build
- Controlling container privilege
- Encrypting inter-microservice traffic
These are worthy topics and they encompass many of the issues that are relevant to securing modern microservice architectures. But there are a couple of important items that I’d like to emphasize:
- Controlling East/West traffic (layers 3 and 4) between Pods within the Kubernetes cluster (aka micro-segmentation)
- Deep inspection of the application traffic (layers 5, 6, and 7) between Pods within the Kubernetes cluster (aka IPS or WAF)
These are concepts that have been around for a while in the traditional on-premises and virtualized data center world. Long ago it was recognized that it was no longer adequate to create a hard, crusty edge and leave a soft, gooey interior for attackers to exploit. The attack surface area can include vulnerabilities buried deep inside the application architecture that can be exploited. These include well-known OWASP top 10 web application attacks such as Cross-site Scripting (XSS), SQL Injection, Remote Code Execution (RCE), API attacks, and more.
Let’s discuss some microservice security patterns that can help.
Kubernetes Application Security Patterns
There are three fairly intuitive design patterns that I will be describing:
- Security Service Layer Pattern
- Security Sidecar Pattern
- Service Mesh Security Plugin Pattern
I’ll be using the following simple Kubernetes deployment to show how we can layer micro-segmentation and application inspection within the cluster to provide better microservice security.
Figure 1: Simple Simulated Microservice Deployment
In this microservice architecture we see three simulated services:
- Public Web interface service
- Internal Authentication service
- Internal Database service
I’m using my microservice traffic and attack generation simulator called microsim to provide a realistic environment with a majority of ‘normal’ JSON/HTTP traffic between services with occasional SQL Injection attack traffic from the WWW service to the internal Auth and DB services.
Now let’s get into the different design patterns.
Security Service Layer Pattern
The Security Service Layer Pattern is probably the simplest to understand, since it is analogous to how micro-segmentation and deep inspection are deployed in traditional environments.
Figure 2: Security Service Layer Pattern
In this design pattern we see the insertion of a security layer in front of each microservice. In this case we are using the official OWASP modsecurity-crs container on Docker Hub. This container provides WAF functionality with the OWASP Core Rule Set and will detect attacks over HTTP, including the simulated SQL Injection attack traffic between microservices. Layer 3 and 4 micro-segmentation is implemented via a network provider that supports Network Policy.
Some of the pros and cons of this design include:
Pros:
- Simple to understand
- Allows scaling of the security tiers independent of the microservices they are protecting
- Treats application security as a microservice
- No need to change microservice ports
Cons:
- Creates additional services in the cluster
- Adds traffic flow complexity
- Requires more micro-segmentation rules
Security Sidecar Pattern
The Security Sidecar Pattern takes the concept of the Security Service Layer Pattern and collapses the additional services into the microservice Pods. Sidecar proxy containers, such as modsecurity, can be explicitly configured as part of the Deployment spec or can be injected into the Pods via MutatingWebhook.
Figure 3: Security Sidecar Pattern
In this design pattern we see the insertion of a security proxy container within each Pod, so both the security proxy and application containers are running in the same Pod. In this case we are also using the official OWASP modsecurity-crs container on Docker Hub. Layer 3 and 4 micro-segmentation is implemented via a network provider that supports Network Policy.
Some of the pros and cons of this design include:
Pros:
- Simple to understand
- Unifies the scaling of the security and application microservices
- The security proxy can be automatically injected into the Pod
- Works with an existing Service Mesh using the Sidecar on Sidecar pattern
- Requires fewer micro-segmentation rules
Cons:
- Requires the Security container and Application container to run on different TCP ports within the Pod
- May result in over-provisioning of the security layer resources
Service Mesh Security Plugin Pattern
The Service Mesh Security Plugin Pattern takes the concept of the Security Sidecar Pattern but implements the security functionality as a plugin to the Service Mesh’s data plane sidecar container (e.g. Envoy in an Istio Service Mesh).
Figure 4: Service Mesh Security Plugin Pattern
In this design pattern we see the insertion of a service mesh data plane container (e.g. Envoy) within each Pod, so both the Service Mesh proxy and application containers are running in the same Pod. In this case the application layer inspection is handled through a modsecurity plugin for Envoy. Layer 3 and 4 micro-segmentation is implemented via the Service Mesh policy.
Some of the pros and cons of this design include:
Pros:
- More cleanly extends security into an existing Service Mesh
- Unifies the scaling of the security and application microservices
- The Service Mesh proxy can be automatically injected into the Pod
- Micro-segmentation rules can be implemented via Service Mesh policy
- Service Mesh enables many advanced application delivery features
Cons:
- Service Mesh deployments are more complex
- May result in over-provisioning of the security layer resources
Secure Microservice POC Deployments
In my opinion, the Security Sidecar Pattern is the most convenient for small projects, but using a Service Mesh is probably a better idea for larger, more complex architectures. In some cases a hybrid approach will make more sense.
Leave a reply if you know of any other designs you’ve seen in the field! Stay tuned for future posts where I will demonstrate simple proof of concept implementations of these security design patterns.
Next in the series: Part 2