KubeCon US 2019 Recap - Open Policy Agent

By Torin Sandall

Torin Sandall

A few weeks ago San Diego hosted the largest KubeCon ever with nearly 12,000 attendees. Let’s take a look at some OPA highlights!

OPA Summit 2019

Day-0 (the day before the Kubecon main event) marked a big milestone for the project: we held our first-ever OPA summit!

Tim Hinrichs shows off the new website to 100+ attendees at the event.

The goal of this summit was to showcase a variety of use cases from companies running OPA in production. We received around 20 submissions to the CFP and then selected a handful of talks that demonstrated impressive scale or new applications for OPA in production.

During the talk about how Pinterest uses OPA, Jeremy Krach and William Fu shared how they architected policy distribution and management for authorization on bare EC2 instances and multi-tenant Kubernetes clusters at Pinterest. Their talk walks through the entire policy lifecycle from authoring to distribution to enforcement. After covering the pipeline, they explained exactly how they offload policy decision-making from services. In terms of volume, their Kafka integration sees the highest traffic. At peak, OPA serves ~450K decisions/second across their clusters (and with caching that increases to ~8.5M decisions/second globally).

William Fu and Jeremy Krach from Pinterest share details on authorization scale at Pinterest.

Michael Sorens from Chef spoke about how they use OPA to implement IAM in Chef Automate. Michael highlighted OPA’s TDD-based approach to policy authoring and explained how they use OPA t implement pre-flight authorization checks that control which UI components are rendered based on user permissions. This use case is becoming more common as OPA is increasingly used higher up in the stack.

Michael speaks about applying TDD to policy.

You can find other great OPA Summit talks from Atlassian, Trip Advisor, and Capital One on YouTube. We are looking forward to hosting the next OPA Summit in Boston at KubeCon US 2020!

Kubernetes: the guardrail enforcement point

At KubeCon after the day-0 summit, engineers from companies like Yelp, Goldman Sachs, Reddit, Adobe, Google, and Microsoft spoke about how they use OPA.

There were several excellent sessions about OPA Gatekeeper and admission control use cases in Kubernetes. OPA is used extensively to enforce guardrails over compute, network, and storage resources in Kubernetes and this was reflected at KubeCon.

Two talks that highlight how Kubernetes is becoming the defacto standard for managing desired state were from Rita Zhang (@ritazzhang) and Ivan Sim (@ihcsim) from Microsoft and Buoyant (respectively) and Sandeep Parikh (@crcsmnky) from Google. This is great for platform administrators because it means they can leverage OPA Gatekeeper to enforce guardrails across not just native Kubernetes resources (e.g., Pods, Services, etc.) but also service mesh resources (e.g., Linkerd, Istio), CI/CD resources (e.g., tekton.dev), cloud resources (e.g., crossplane.io), and more.

Miguel Uzcategui (Goldman Sachs) and OPA co-founder Tim Hinrichs (CTO of Styra) spoke about how Goldman Sachs uses OPA to do policy-based provisioning in Kubernetes. They explain how Goldman Sachs implemented Kubernetes controllers that offload decision-making to OPA so that when namespaces are created, resources like quotas, roles, and persistent volumes (and claims) are automatically instantiated (and then re-converged if something changes.) This is important for maintaining strict requirements around security and availability at Goldman Sachs. They also show why OPA is a good fit for this problem (e.g., easier testing, ability to use external context in decision-making, etc.) and how it has performed in production for nearly a year.

Miguel shares results from running the OPA in production for over a year.

Outside Kubernetes: App configuration and Microservices

@garethr’s talk about applying OPA policies earlier in application lifecycles highlighted OPA’s general-purpose nature. His talk was filled with examples and demos that show how to use OPA and conftest to validate configuration files (e.g., Pipfiles, Dockerfiles, etc.) and plug into CI/CD systems.

Finally, on Thursday, Daniel Popescu and Ben Plotnick talked about how Yelp evolved their security infrastructure (using OPA and Envoy) as the company transitioned away from a monolith. Their talk highlights how perimeter-based security does not scale to microservice architectures and how development of custom policy languages is challenging. The talk provides a deep dive on how they leverage Envoy and OPA to implement mTLS and access control across a fleet of microservices. They also discuss the gradual migration off their custom policy language by transpiling to OPA.

This post only highlights a few of the excellent talks from KubeCon about OPA so if you want to watch more check out this playlist on YouTube.

Conclusions

KubeCon San Diego demonstrated how many companies run OPA in production for a variety of use cases. After starting the project nearly four years ago it is very exciting to see it fulfilling the original goal of modernizing and enabling policy enforcement across the stack.

Last year we saw rapid growth in end-user adoption, the launch of OPA Gatekeeper, and promotion to the CNCF Incubating tier. In 2020 we plan to continue investing in performance and usability for the core of the project, new integrations leveraging the recent WebAssembly compiler feature, and better documentation of reference architectures.

See you all in Amsterdam and Boston!