FOSDEM is a free event for software developers to meet, share ideas and collaborate. Every year, thousands of developers of free and open source software from all over the world gather at the event in Brussels.
The State of Go
What’s new since Go 1.12
Francesc Campoy & Maartje Eyskens
Go 1.14 is planned to be released in February 2020- and this talk covers what’s coming up with it. We’ll talk about new features and fixes in Go. All of the new things you might have missed.
Way to many changes to add them to my notes. Just check the slides.
GopherCon Europe 2020, April 23-26 2020 Berlin
Mario’s adventures in Tekton land
Testing, releasing and deploying Tekton with Tekton
Andrea Frittoli | IBM Vincent Demeester | Red Hat
Tekton is a Kubernetes-native, lightweight, easy to manage CI/CD pipelines engine. Pipeline building blocks can be reused, version controlled and curated in a catalogue that embeds best practices.
Continuous Delivery Foundation (CDF)
The CDF is a neutral home for the next generation of Continuous Delivery collaboration.
The Continuous Delivery Foundation (CDF) serves as the vendor-neutral home of many of the fastest-growing projects for continuous delivery, including Jenkins, Jenkins X, Spinnaker, and Tekton. CDF supports DevOps practitioners with an open model, training, industry guidelines, and a portability focus.
“As the market has shifted to containerized and cloud native technologies, the ecosystem of CI/CD systems and related tools has radically changed. The number of available CD tools has increased, and there’s no defining industry specifications around pipelines and workflows to aid portability amongst tools. Capital One, CircleCI, CloudBees, Google, Huawei, IBM, JFrog, Netflix and the other Continuous Delivery Foundation (CDF) founding members recognize the need for a neutral home for collaboration and integration to solve this problem. The CDF will establish a community of projects to advance industry best practices and innovation around CI/CD.”
Chris Aniszczyk, Vice President of Developer Relations, the Linux Foundation
Current CD projects:
- Jenkins
- JenkinsX, Jenkins for Cloud Native and Kubernetes environments. All you need to know about Jenkins X
- Spinnaker
- Tekton
- Screwdriver.cd has status incubating
Tekton in a nutshell
- Standard Kubernetes-style pipelines
- Run pipelines in containers
- Build images with Kubernetes tools
- Deploy to multiple platforms
- Powerful command-line tool
Tekton pipeline concept
- Step, run commands in a container with volumes, env vars etc
- Task, a list of steps that run sequentially in the same pod
- Pipeline, a graph of tasks with inputs and outputs executed in a certain order
- PipelineResource, inputs and outputs to tasks and pipelines (git, image, eyc.)
- TaskRun, an invocation of a task with inputs and outputs
- PipelineRun, an invocation of a pipeline with inputs and outputs
Initial plumbing
Tekton inherits its plumbing from Knative
- Infrastructure on Google cloud
- Heavy usage of Prow
- Sharing scripts and container mages with Knative
Kubernetes & Knative use test-infra, we thought plumbing was more fun ;-)
Prow
Prow is a Kubernetes based CI/CD system. Jobs can be triggered by various types of events and report their status to many different services. In addition to job execution, Prow provides GitHub automation in the form of policy enforcement, chat-ops via /foo style commands, and automatic PR merging.
Prow is used by many organizations and projects; Kubernetes, OpenShift, Istio, Knative, Metal3, Prometheus, JenkinsX etc. etc.
Gubernator
Gubernator is a frontend for displaying Kubernetes test results stored in GCS.
It runs on Google App Engine, and parses JSON and junit.xml results for display. https://gubernator.k8s.io/
ko
ko is a tool for building and deploying Go(lang) applications to Kubernetes github/google/ko.
Tide
Tide is a Prow component for managing a pool of GitHub PRs that match a given set of criteria. It will automatically retest PRs that meet the criteria (“tide comes in”) and automatically merge them when they have up-to-date passing test results (“tide goes out”).
Tide used for triggering builds based on label.
Goals
- Step 0 - Become completely independant of Knative
- Step 1 - Build Tekton with Tekton
- Step 2 - Get Tekton on par with Prow functionality
- Step 3 - Dogfooding cluster and become independant of Prow
- Step 4 - Incremetal steps to CD
- Step 5 - Missing pipes
Note: An actual palindrome day: 02.02.2020
Verdict: Promising but incomplete
Test Software On Emulated Hardware In Containers… In The Cloud
Sean A. Parker, Paul Schroeder | Accenture | Fosdem 0x7e4
Main goal is automating and modernizing traveller information on very different hardware devices for a large railway infrastructure.
Move from decentralised legacy system to a centralised system.
Many different peripherals but aiming for “one” buildroot.
QEMU
QEMU is a generic and open source machine emulator and virtualizer.
QEMU has two operating modes:
- Full system emulation. In this mode, QEMU emulates a full system, including one or several processors and various peripherals.
- User mode emulation. In this mode, QEMU can launch processes compiled for one CPU on another CPU. It can be used to ease cross-compilation and cross-debugging.
noVNC
noVNC - the open source VNC client - it’s is both a VNC client JavaScript library as well as an application built on top of that library. noVNC runs well in any modern browser including mobile browsers (iOS and Android).
Qocker
Running QEMU in Docker.
Recap
- Testing our OS with different hardware configurations on actual hardware is very hard
- QEMU allows automatic simulation of hardware
- Mocking hardware components is a way out but no silver bullet
- QEMU and Docker allow relatively easy testing in the CI at scale
From Go to Kubernetes CRDs and Back
Workflow for building strongly typed APIs
Stefan Schimanski
To late for this session but there is a video recording 😉
Choosing The Right Deployment Strategy
Viktor Farcic | @vfarcic | Technology Conversations | devopstoolkitseries | DevOps Paradox podcast
Session was full, unable to attend but … there are slides (in Reveal.js) and a video recording
LXD for mixed system containers and VM workloads
Introducing LXD’s new virtual machine feature
Stéphane Graber | Canonical | 12:30
Classify things in Go: the easy way
Building classifiers quickly with the community contributions
Sheimy Rahman | @SheimyRahman | github/SheimyRahman | Nuveo.ai
Computer Vision
Computer Vision (CV) is the ability of computers to process visual information, and perform tasks normally associated with those performed by humans.
CV software typically processes video images, then uses the data to extract information in order to do something useful.
GoCV
GoCV gives programmers who use the Go programming languageaccess to the OpenCV 4 computer vision library.
The GoCV package supports the latest releases of Go and OpenCV v4.2.0 on Linux, macOS, and Windows. We intend to make the Go language a “first-class” client compatible with the latest developments in the OpenCV ecosystem.
GoCV also supports the Intel OpenVINO toolkit.
Just looking for the code? Go to https://github.com/hybridgroup/gocv
Official online live training “Getting Started with Computer Vision Using Go”
TensorFlow Hub
TensorFlow Hub] is a library for the publication, discovery, and consumption of reusable parts of machine learning models. A module is a self-contained piece of a TensorFlow graph, along with its weights and assets, that can be reused across different tasks in a process known as transfer learning.
Transfer learning can:
- Train a model with a smaller dataset,
- Improve generalization, and
- Speed up training.
Caffe
Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR) and by community contributors. Yangqing Jia created the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license.
Check out our web image classification demo!
During the demo something went wrong with the sound …
But the demo worked!
Advanced debugging techniques of Go code
Andrii Soldatenko | @a_soldatenko
F0SD3M 02022020 session info | Slides
Go debuggers:
- Delve
- GDB, has issues on OSX
- Ogle, deprecated?
Current versions of Go produce DWARF. DWARF is a debugging file format used by many compilers and debuggers to support source level debugging.
Disable DWARF generation without loss of functionality:
go build -ldflags=-w
Delve seems to be the debugger tool of choice.
Delve
Delve a Debugger for the Go Programming Language.
delve: 101
dlv debug- Compile and begin debugging main package in current directory, or the package specifieddlv test- Compile test binary and begin debugging program.DWARF specification
Debugging Go code using VS Code or VSCodium 😉
Debugging containerized Go apps
Problem:
$ docker run -it --rm my-golang-app bash
$root@03c1977b1063:/go/src/app# dlv debug
main.go
could not launch process: fork/exec /go/src/app/
__debug_bin: operation not permitted
Solved:
$ docker run -it --rm —security-opt="apparmor=unconfined" —cap-add=SYS_PTRACE my-golang-app bash
$ root@7dc3a7e8b3fc:/go/src/app# dlv debug
main.go
Type 'help' for list of commands.
(dlv)
AppArmor security profiles for Docker
Configure Delve
Two options to configure Delve:
$HOME/.dlv/config.yml(dlv) config -list
Future Reading
- Internal Architecture of Delve - slides
- DWARF specification DWARF
- Delve documentation
(dlv) help- source code of go
Baremetal at the Edge
Managing bare metal machines where PXE would fail
Ilya Etingof | Red Hat
Why PXE-boot is unreliable?
- IPMI, DHCP, TFTP & PXE protocols need several package exchanges
- DHCP requires L2 connectivity by design
- Boot image transfer over TFTP is unreliable
- Security: image & node identification is hard to implement
A new way of booting
- Redfish to replace IPMI and vendor-specific protocols
- Virtual media can replace PXE/TFTP and DHCP
OOB node configuration
- With PXE nodes are hardly distinguishable
- Virtual media as an OOB configuration method
- Static network configuration
- Secrets
Demo
Booting Ironic ramdisk over virtual media. We’ll be using operator specified network configuration instead of DHCP.
Redfish
DMTF’s Redfish® is a standard designed to deliver simple and secure management for converged, hybrid IT and the Software Defined Data Center (SDDC). Both human readable and machine capable, Redfish leverages common Internet and web services standards to expose information directly to the modern tool chain.
Summary
- Edge cloud drives further innovation
- Redfish is the future of hardware management
- Open Source projects are heavy players
A VM journey from VMware to Kubernetes
Marek Libra | Red Hat
Kubevirt addon to Kubernetes, run Virtual Machines as / alongside containers.
The session was to show how to migrate from VMware to Kubernetes / OKD. “VMware baaad, Red Hat gooood!”
Running virtual machines out of thin air
Nir Soffer | nsoffer@redhat.com | Red Hat Daniel Erez | derez@redhat.com | Red Hat
oVirt
oVirt is an open-source distributed virtualization solution, designed to manage your entire enterprise infrastructure. oVirt uses the trusted KVM hypervisor and is built upon several other community projects, including libvirt, Gluster, PatternFly, and Ansible.
Gluster
Gluster is a free and open source software scalable network filesystem.
GlusterFS is a scalable network filesystem suitable for data-intensive tasks such as cloud storage and media streaming. GlusterFS is free and open source software and can utilize common off-the-shelf hardware.
Patternfly
PatternFly is an open source design system built to drive consistency and unify teams. We provide tools like design documentation, components, and code examples to make it possible for anyone to design and build responsive, accessible web applications.
oVirt supports a lot of storage types
- NFS
- Gluster
- iSCSI
- Fibre Channel
- Cinder storage
- Local storage
Running oVirt with storage on external system
QEMU supports network disk types:
- iSCSI
- NBD, Network Block Device
- SSH
- HTTP
- HTTPS
- FTP
- FTPS
Running a VM without any disk!
qemu-kvm nbd://external.com/sda
See the video recording for a demo of booting from a remote disk. Truly hacking oVirt.
More info
- oVirt patches in review: https://gerrit.ovirt.org/q/topic:external-disk
- Demos: https://drive.google.com/drive/folders/1LmILHx7urCB9hOYFvU4V8vn5ulbwOFy3
- QEMU manual: https://qemu.weilnetz.de/doc/qemu-doc.html#Device-URL-Syntax
Nerdy session but very interesting concept!
Speed up the monolith
building a smart reverse proxy in Go
Alessio Caiazza | @nolith | alessio@gitlab.com | GitLab
Workhorse a smart reverse proxy.
FOSSH - 2000 to 2020 and beyond!
maddog continues to pontificate
maddog reviews the past twenty years of FOSSH, and dares to predict the next twenty years.
FOSSH is the abbreviation for Free and Open Source Software and Hardware.
Why Open ?
- Developers know immediately through standard licenses if they can use the code
- Developers do not have to make source code avaialable to end users
Why Free(dom)?
- Developers and End Users have access to source code
- Make bug fixes
- Add features
- Even if end users do not have expertise
Jon refers to a drawer full of old devices that run on non-free software and can not be be fixed because the software is not available.
Expertise or not the end users do not have the choice because the source is not available. Another example; If the source code for Windows XP would have been available it could have been patched …
What Are New Challenges?
- Who really owns Free Software?
- Companies?
- Developers?
- Community?
- End Users?
- All of the above?
- Companies now producing “closed solutions” with “Open Source”
- We have 95% users who don’t know abouit freedom (or care if they know about it).
More New Challenges … and Opportunities
- Security and Privacy - it is worse than most people suspect
- Clouds in our homes or business - Freedombox
- Ease of use
- “Mom&Pop™” need help
- AI - should be renamed “Inorganic Intelligence”
- People keep saying “retrain the displaced” ….
- Money vs Community - money for “advertising”
- Education with FOSSH
- FOSSH teaches three times
The Fight Is Not Over
- Love is Love
- Steve Balmer now says he loves Open Source
- Many companies say the love Open Source
- End users shouls love freedom Software
- Some entities are (very vocall) moving from FOSSH (back) to closed-source proprietary code
- We (some of us) are getting old(er) ….
- Mentorship more important than ever …
- “I listened to you 10, 20 ,30 ,40, 50 years ago…
- Pay it forward ….
“History” is still being created… not a leap, but ever-increasingley fast changes…
If You Want to See the Most Important Person in FOSSH….
Look in the mirror…
…next year bring two MS developers.
Recommended by others
- Debugging apps running in Kubernetes - An overview of the tooling available. FOSDEM session info
- Do Linux Distributions Still Matter with Containers? FOSDFEM session info
- Some Excerpts from the Theory of Design in Architecture. FOSDFEM session info
- From Go to Kubernetes CRDs and Back - Workflow for building strongly typed APIs. FOSDFEM session info
Otherwise interesting
- Fixing healthcare data exchange with decentralized FOSS - Building a decentralized Infrastructure to fix medical data exchange in The Netherlands. FOSDFEM session info
- Tarantool Cartridge - Framework for Distributed App. FOSDFEM session info