Kubernetes continues to be a transformative technology in the IT industry. Recently, the Dutch Kubernetes Podcast, hosted by Ronald Kers and Jan Stomphorst, featured an engaging episode with special guests Jurg van Vliet and Maarten Kamoen. In this episode, they discuss their experiences, challenges, and the future of Kubernetes, along with the vision for a sustainable European cloud infrastructure. 

Jump to sections:

• Meet the Guests and Their Backgrounds
• The Evolution and Vision of Kubernetes
• Technical Insights and Innovations
• Future of Kubernetes and the European Cloud

Meet the Guests and Their Backgrounds

Jurg van Vliet is the CEO of Aknostic and a seasoned computer scientist. With a rich background that includes authoring O’Reilly books about AWS, Jurg brings a great deal of practical knowledge and experience. During his years in the tech industry, he has significantly contributed to cloud infrastructures and deeply understands various programming languages, including Java and Scala.

Maarten Kamoen, a Staff Engineer at Aknostic, started his career in Linux and Unix System Administration. His expertise in managing data centers smoothly transitioned into working with Kubernetes. Since 2018, Maarten has been involved with AWS’s EKS and integration challenges and has witnessed the platform's advancements. His practical experience setting up and managing Kubernetes clusters has greatly benefited Aknostic’s approach.

The Evolution and Vision of Kubernetes

Initial Challenges and Current Trends

Maarten shared his early experiences with Kubernetes, particularly the difficulties faced when integrating Kubernetes with AWS. In 2018, the integration between Kubernetes, EKS, and the rest of the AWS ecosystem was limited. Tools, now taken for granted, like the AWS load balancing controller, were unavailable and required significant manual setup. Despite these challenges, the ecosystem quickly became more cohesive and feature-rich. Today, Kubernetes is seen as an indispensable tool in IT infrastructure, simplifying the management of containerized applications and services.

Vision for a European Cloud

During the interview, Jurg elaborated on his work with the Sustainable Digital Infrastructure Alliance (SDIA), focusing on developing a European cloud. Unlike existing clouds like AWS or Azure, this European cloud aims to be a marketplace of compute and storage commodities. The vision involves creating a decentralized infrastructure where various providers contribute resources to a shared marketplace, making it more resilient and adaptable.

A key aspect of this vision is sustainability. Jurg emphasized that the European cloud must lead in sustainability by incorporating transparency in reporting energy usage, water consumption, and embedded emissions. This transparency is not just about being environmentally friendly; it’s about building trust with users and stakeholders. The European cloud can set a new standard for accountability in the tech industry by openly sharing data on resource usage and environmental impact.

This ‘can be’ alternative to suppliers such as AWS and Azure also aims to differentiate itself through innovation. Instead of merely replicating existing cloud models, it seeks to offer unique features that cater to the specific needs of European users and industries. This includes better integration with European regulations, enhanced data privacy, and support for local businesses.

A core condition for this is orchestrated collaboration between service providers, software developers, and policymakers. Jurg believes this approach can help Europe compete more effectively in the global cloud market while promoting values that are increasingly important to consumers and businesses.

Technical Insights and Innovations

Kubernetes Implementation and Hardware Optimization

Maarten and Jurg use open-source components and the Cluster API to manage Kubernetes clusters efficiently. They leverage tools like KubeVirt to manage virtual machines, allowing them to run Kubernetes within Kubernetes and maximizing resource utilization and flexibility.

A key strategy discussed during the podcast was the use of refurbished hardware. By extending the lifespan of existing hardware and integrating it into their infrastructure, Aknostic reduces costs and minimizes environmental impact. This approach aligns with their commitment to sustainability, proving that high performance and sustainability practices can go hand in hand.

The always-returning discussion with this topic is the alleged increased carbon efficiency of new model hardware (chips in particular). This is the ultimate gaslighting move. The point is that the lifespan of stuff is essential. Refurbished can mean re-using the chassis and retrofitting it with later generations of chips, memory, and disks. However, in the end, this debate will only be settled when we have full transparency (embodied emissions) and carbon reporting. In more colloquial terms, perhaps old shit running the Nordics is better than the latest and greatest in a dense urban environment like Amsterdam.

Security and Network Management

Security is a significant concern in multi-tenant Kubernetes environments. Maarten explained the challenges and solutions they implement to ensure robust security. This includes using network policies and isolating data storage to protect against vulnerabilities. Tools like Cilium and Calico are employed for efficient network management, tailored to specific use cases, such as handling Windows workloads, which add another layer of complexity.

Ensuring isolation and security is paramount, particularly in environments where multiple tenants share resources. Aknostic’s approach involves using advanced network configurations and continuous monitoring to maintain a secure and reliable infrastructure.

Of course, for some use cases, more is needed. In these cases, full isolation can be the answer either in the same data center or on-premise.

Future of Kubernetes and the European Cloud

Kubernetes as a Standard API

Both Jurg and Maarten are optimistic about Kubernetes' future. They envision it becoming the standard API for deploying workloads across various environments, from public clouds to on-premise data centers. Kubernetes' flexibility and scalability make it ideal for managing diverse applications and services, ensuring consistency and reliability.

Maarten highlighted that Kubernetes’ role as a standard API could simplify deployment processes and provide a consistent framework for developers. This standardization would enable smoother transitions between different environments, whether on-premise or in the cloud, reducing complexity and increasing efficiency.

Impact of a European Cloud

As discussed earlier, the European cloud can significantly impact the global cloud market by setting new industry standards and offering developers more choices by prioritizing sustainability and transparency. This approach can foster innovation, promote data privacy, and support local businesses, creating a more balanced, innovative, and competitive market. This fairer marketplace, where multiple providers offer services, ensures diversity and resilience, leading to better end-user services and solutions.

Jurg and Maarten believe a collaborative effort within the European tech community is crucial for this vision's success. By leveraging open-source principles and engaging with various stakeholders, the European cloud can become a powerful tool for shaping the future of digital infrastructure.

Furthermore, the European cloud’s emphasis on transparency and sustainability addresses growing concerns about environmental impact and corporate accountability. Setting a benchmark for openness in resource usage reporting can build trust and drive industry-wide changes toward more responsible practices.

Another significant advantage is the potential for the European cloud to integrate seamlessly with existing regulations and data privacy laws. It can provide a trusted environment for European businesses, ensuring compliance with stringent data protection standards and offering a reliable alternative to non-European cloud providers.

The European cloud represents a forward-thinking approach to cloud infrastructure, combining technological innovation with a solid commitment to sustainability and transparency. It can redefine the cloud landscape, offering new opportunities and setting higher standards for the industry.

Aknostic is investing heavily in sustainability insights. With the KEIT (Kubernetes Emissions, Insights Tools) project, we aim to put emissions (not just energy) reporting into the hands of developers. With these insights, architectural choices can be made with more complete information. It should also make it easier to shift the load closer to the core of Kubernetes and introduce carbon resource limits (trade carbon for latency). The biggest challenge is complete transparency from data centers to equipment manufacturers.

This is just the beginning.

This discussion on the Dutch Kubernetes Podcast highlighted Kubernetes' transformative potential and the vision for a sustainable European cloud. Jurg van Vliet and Maarten Kamoen’s insights into the challenges, innovations, and future directions provided valuable perspectives for anyone interested in cloud infrastructure and sustainability.

Share your thoughts and experiences in the comments below, and let’s explore how we can collectively shape a more sustainable and transparent digital future.

Follow the Dutch Kubernetes Podcast for more in-depth discussions and insights.