There isn’t a single book or article on observability (O11y) where there isn’t a mention of MELT. (Metrics, Events, Traces, Logs) While these four are the building blocks of telemetry data in Observability, all the four components haven’t evolved at the same rate. In this write-up, I delve into metrics in observability (mainly custom metrics) and argue how the overuse of metrics is turning into madness and directly impacting the cost of observability. ...

In distributed systems, the principle of message passing between nodes is a core concept. But this leads to an inevitable question: How can we ensure that a message was successfully delivered to its destination? To address this, there are three types of delivery semantics commonly employed: • At Most Once • At Least Once • Exactly Once Each of these offers different guarantees and trade-offs when it comes to message delivery. Let’s break down each one: ...

This post is a geek-out journey this festive season on finding the right tool for my personal finance management. Where it all started: Recently, I spoke at the Smarsh Tech Summit on “Cost as an Architectural Pillar,” where I emphasized the importance of considering cost as a first-class citizen in the software development cycle. However, on a similar note, when I was looking through my personal finances later, I wasn’t very happy when I had to apply the same principle. I was using one of the apps for finance management, and it was all over the place. Since it is festive time off at work, I started looking around for the best way to fix this and track my personal finance the right way. ...

This is a brain dump on how taking notes and Obsidian as a tool has helped me. Knowledge management? As one progresses further into career, Knowledge management becomes as equally important as Finance management. Knowledge accumulation is a non-linear trajectory. Majority of the times it is compounding in nature. If one doesn’t organise it, you are always at the mercy of “I had solved this once before but don’t remember how”. ...

This writeup is an outcome of a side quest while geeking out on System Design. In the book “Designing Data-Intensive Applications,” Bloom Filters are briefly mentioned in the context of datastores, highlighting their significance in preventing database slowness caused by lookup for nonexistent items. Below are a curious set of questions on the topic on Bloom Filters and how it works. What is the use case for a Bloom filter? Imagine you are maintaining a Datastore which has millions of records. You want to search for an item form the Datastore, while you are not sure that the item exists in the first place. ...

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : The cost crisis in the observability space is a real problem. Here is an article that describes the issue: - link here How many conferences are too many? Here is an exhaustive list of all the popular talks on Kubernetes from 2023: - link here OpenTelemetry is the industry standard in observability. Here is a list of anti-patterns with observability to avoid:- link here ...

Over the past few years, I have worked on different multi-cluster distributed datastores and messaging systems like - ElasticSearch, MongoDB, Kafka etc. From the Platform Engineering/SRE perspective, I have seen multiple incidents with different distributed datastores/messaging systems. Typical ones being : uneven node densities (ElasticSearch - how are you creating shards?) client node issues (client/router saturation is a real thing. And they need to be HA) replicas falling being masters, (Mongo - I see you) ...

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : System Design - Designing a Ticket Booking Site Like Ticketmaster is the most common system design question - link UberEngineering blog on Anomaly detection and alerting system - link P99 CONF 2023 | Always-on Profiling of All Linux Threads by Tanel Poder - YouTube link On choosing Golang as a programming language at American Express- link ...

In this write up, we will try and explore how to make the most out of the resources in K8s cluster for the Pods on them. Resource Types: When it comes to resources on Kubernetes cluster, they can be fairly divided in to two categories: compressible: If the usage of this resource for an application goes beyond the max, it can be throttled without directly killing the application/process. example : cpu - if a container consumes too much of compressible resource, they are throttled ...

[Self Notes and Review]: This is a second writeup in the series of reading DDIA book and publishing my notes from the book. The first one can be found here This particular article is from the second chapter of the book. Again, these are just my self notes/extracts and treat this more like an overview/summary. Best way is to read the book in itself. This chapter dwells in to the details of: the format in which we write data to databases and mechanism by which we read it back. ...