Open Source Summit Europe 2018
Open Source Summit Europe is the leading conference for developers, architects and other technologists – as well as open source community and industry leaders – to collaborate, share information, learn about the latest technologies and gain a competitive advantage by using innovative open solutions.
This page contains my personal notes of the first conference day.
Wanted, 10,000 Developers to Electrify the Planet
Shuli Goodman, Executive Director, LF Energy
Energy: distrbuted everything
Microsoft Participation and the Further Reduction of Patent Risk in the Core of Linux/OSS
Keith Bergelt, Chief Executive Officer, Open Invention Network (OIN)
OpenInventionNetwork: Microsoft Participation and the Further Reduction of Patent Risk in the Core of Linux/OSS
OIN Community, join to share your patents.
Something for KPN, to join?
Software-Defined Everything
Arjan van de Ven, Intel Fellow & Director of Linux Systems Engineering, Open Source Technology Center, Intel
Software-Defined-Everything
ACRN
ACRN™ is a flexible, lightweight reference hypervisor, built with real-time and safety-criticality in mind, optimized to streamline embedded development through an open source platform
ACRN, functionally safe capable hypervisor
Kata containers
Kata Containers is a new open source project building extremely lightweight virtual machines that seamlessly plug into the containers ecosystem.
Nemu
NEMU is an open source hypervisor specifically built and designed to run modern cloud workloads on modern 64-bit Intel and ARM CPUs.
Modern Open Source Hypervisor for the Cloud, QEMU based.
The Kernel Report
Jonathan Corbet, Author, Kernel Developer and Executive Editor
Handling of hardware vulnerabilities: The look like software vulnerabilities. Can be solved in software but requires openness.
Meltdown was developed in public, Spectre was developed in private. Spectre solution:
- distributor fragmented
- developer burnout and frustration
- many left out in the cold
Current stable kernel: 4.18.16
The Good Olde Days:
- soource-code management
- change tracking
- release discipline
- rules on regressions
- automated testing
- …
- code of conduct
AIOps: Anomaly Detection with Prometheus
Marcel Hild, Red Hat
Prometheus, fire to the people.
Pulling data, long term storage and alerting
Prometheus => monitoring & alerting
How Red Hat sees AI
- Project Thoth and Bots, representa a workload requireemnt for our platforms across the hybriid cloud
- This talk
- OpenDataHub, data as the foundation
Thanos:
- Prometheus at scale
- Global query view
- Reliable historical data storage
- Unlimited retention
- Downsampling
InfluxDB:
- Easily hooked into prometheus with write and read endpoint
- Reliable historical data storage
- Great for data science
- Pandas integration
- “Eats RAM for breakfast”
- GitHub: Prometheus Long Term Storage
Prometheus scraper:
- container can be configured to scrape any prometheus server
- can scrape all or a subset of the metrics
- stores data in ceph or S3 compliant storage
- can be queried with spark sql
- Future Proof: path to Thanos
- GitHub: Store prometheus metrics with thanos
Spark:
- Query stored JSON files
- Distribute the workload
- Use spark library
- GitHub-notebook:Verify Alerts with Spark and Explanatory Statistics - CEPH.ipynb
Analysis of metrics meta data: GitHub: Data Science on Prometheus Metrics
Anomaly Types
- Irregularity
- Seasonality
- Increasing trend
Anomaly detection with Prophet, predicting future data and dynamic thresholds.
- ‘list_images’ operation
- on OpenShift
- monitored by Prometheus
- upward trends
- intraday seasonality
- GitHub-notebook: Prophet Model Forecasting.ipynb
The Accumulator, GitHub-notebook: Anomaly Detection Decision Rules.ipynb
GitHub: Anomaly Detection in Prometheus Metrics
Demo: MiniShift (Openshift) Fourier and Prophet (anomaly times two, expected divided by two)
CRI Runtimes: Who is running my Pod!? A Deep Dive into Kubernetes and the Container Runtime Interface
Phil Estes, @estesp, IBM
Kubernetes doesn’t run your containers
Kubernetes vs Container Runtime
What Runtimes Exist?
What do I need?
- Performance
- Stability
- (Optional) Hypervisor Isolation
- Security Capabilites
- Broad usage
- Multi-architecture Support
Containerd Benefits
- Designed with broad usage as a core container runtime:
- Docker, LinuxKit, Kubernetes and embedded core runtime use cases (OpenWhisk, Cloud Foundry)
- Stress testing for stability and performance guarantees 24/7
- Usable Go librar (or gRPC) for ease of embedding
- Compatibility guarantees; bug fix backports for stable support
Containerd in the Cloud(s)
- Kelsey Hightower’s “Kubernetes the Hard Way” deploys containerd as the kubernetes runtime
- GKE beta: containerd-based K8s clusters
- IBM Cloud: containerd-based clusters for 1.11+
- Azure OSS acs-engine includes containerd; AKS moving to containerd (but CRI-O for OpenShift)
- CloudFoundry: moving to containerd from runc
Going Further
- crictl User’s Guide
- Stephen Day’s KubeCon 2018 containerd Deep Dive talk
- Containerd project
- containerd.io
containerd is an industry-standard container runtime with an emphasis on simplicity, robustness and portability. It is available as a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc.
containerd is designed to be embedded into a larger system, rather than being used directly by developers or end-users.
crictl & ctr
Read the user’s guide
ctr version displays containerD version
ctr has namespaces
ctr can be used to load a container image into the container runtime
OpenFaaS Cloud: Multi-user Serverless driven by GitOps
Alex Ellis, alex@openfaas.com, OpenFaaS project
Fulltime team, Production users, Won a Bossie award (twice)
Serverless is an architectural pattern
Cloud: decreasing concern for infrastructure
Application evolution
Function definition
Function:
- short lived
- single-purpose
- no state
- auto-scaling
- Why not atomic
Language support
built-in support for several languages:
- Go
- .NET core
- Python
- etc
Github Actions
GitOps + functions = OpenFaas Cloud:
- Managed
- Multi-user
- Personal dashboard
- CI/CD (GitOps)
- Feedback via GitHub
Summing up
OpenFaas Cloud brings:
- Portable multi-usert Serverless
- Developer-native workflow (GitOps)
- Rapid iteration from concept to production