Fosdem 2023

fosdem logo

This year has started with the FOSDEM conference in Brussels. I’ve had only one goal - to boost my technical skills with the newest technologies and technics. I spent the 1st day in Golang room and the 2nd in Monitoring & Observability. After 2 very intense days I calmly arrived home and started catching up with other rooms. This is how the post appeared. Enjoy! 🌟

👀 - watch
📚 - read

Go room

👀 The State of Go

a more convenient way to convert a slice to an array
improvements to generics - comparison
toolchain improvements - introduced new commands, improved go vet
changes in the standard library - error(multiple errors wrapping), time, bytes & strings
Go future: Go 2.0 and drafts - structured logging, go version compatibility (metrics)

👀 Recipes for reducing cognitive load

line of sight
better errors
pure functions
mysterious booleans
package splitting, public objects on the top

📚 Building a CI pipeline with Dagger in Go

Dagger has few SDKs available: Go, Python, etc.
Makefile for Go - mage
write pipelines with Go, use all benefits of the language
supported container runtime - docker 🙄

📚 Debugging concurrency programs in Go

colored output
debuggers - delve, gdb
profile labels, goroutines labels
debugger middleware

📚 Optimizing string usage in Go programs

3 things:

understanding go strings behind the scenes
string use cases prone to performance bottlenecks
optimization strategies:
- detaching strings from larger memory pools
- string interning
- string interning with symbol table
- string concatenation

👀 Squeezing a go function

benchmarks
profiling
reducing CPU usage
reducing allocations
packing
function inlining
escape analysis
concurrency

👀 Five Steps to Make Your Go Code Faster & More Efficient

Use Test Fix Benchmark Optimize
Understand current efficiency level (micro-benchmarks)
Understand your efficiency requirements
Do profiling
Try to optimize and repeat

👀 Our Mad Journey of Building a Vector Database in Go

memory allocations are important for high-performance applications
- binary.Read() isn’t optimal
- stack is short-lived, fast, and chip, heap is long-lived, expensive and slower
- binary.LittleEndian.Uint32 allows reading without memory allocations
delayed decoding
- json is too slow
- binary encoding is fast, but decoding everything up-front is still expensive
- replace allocation with summing up
- in Go 1.20 experimental support for memory arena that potentially can be used to bypass the GC
SIMD (single instruction, multiple data)
- assembly allows to do it
- there is no way to write explicit SIMD instructions in pure Go
- avo allows to generate x86 assembly

👀 Building a basic event-driven application in Go in 20 minutes. Introduction to Watermill

watermill allows simplifying writing event-driven apps
provides an easy way to test things
https://threedots.tech/fosdem2023/

Monitoring & Observability

📚 Monitor your databases with Open Source tools

PMM - various DB(postgres, mysql, mongodb) monitoring, query monitoring

👀 Observability in Postgres

a lot of useful data in the internal DBs
there is a prometheus-exporter with support for custom queries
it has issues e.g. performance, no metrics if running out of max_connections, …
can be improved by Postgres itself by introducing support for monitoring via a port.

👀 Loki: Logging, but make it cloud native

👀 Adopting continuous-profiling

interesting approach to have observability to debug high-performance applications
Phlare is a Grafana open-source solution (other: pixie, parca, pyroscope)

👀 Inspektor Gadget: An eBPF-based Tool to Observe Containers