Observability on Akshay Deshpande

Metrics Types - When to use what ?

Thu, 30 Apr 2026 00:00:00 +0000

This writeup talks about different metric type that an application can emit as telemetry. It intends to cover the case of when to use which metric types and the usecase for them.

Metrics is one of the oldest forms of telemetry. There are many APM solutions, whose billing model runs on the number of metrics you send to them. But it is often not clear to app devs on what kind of metrics to emit from applications. Infact, the types of metrics can be confusing sometimes.

Prometheus: Static/Dynamic scraping on EKS

Thu, 20 Nov 2025 00:00:00 +0000

This note is a mental model for how Prometheus discovers and scrapes metrics in Kubernetes.

The lens I want to keep throughout is:

Where will the scrape config file sit?
(Prometheus repo vs application repo)
In which namespace will the serviceMonitor sit?
(and how Prometheus finds it)

At a high level there are two ways to tell Prometheus about a /metrics endpoint:

Static via in the Prometheus config file.
Dynamic via (CRD from Prometheus Operator) with label‑based discovery.

Kubecon India : 2025

Thu, 25 Sep 2025 00:00:00 +0000

I attended Kubecon India 2025 held in Hyderabad this year. I mainly focused and attended talks related to Observability and Scalable designs.

Now that the sessions are uploaded to Youtube, linking the ones which I really enjoyed.

Observability at Scale With Monitoring as Code: Grafana, Prometheus, & Tempo - Vipin GopalaKrishnapillai & Saiabhinay Bommakanti, Amway Global - link
Predictable auto scaling with keda - link
Observability - tenant centric metrics - link
Building observability platform for Edge compute nodes - link

Design Philosophy: Observability

Sun, 14 Sep 2025 00:00:00 +0000

I enjoy philosophy. Stoic philosophy in particular.
Philosophy, I think, helps us revalidate our purpose. It acts as a yard stick and makes sure that we are not moving away from our First-Principles.

Applying the same to Software Engineering, in my opinion, every team should have a “Design Philosophy”. What is that one yard stick which teams can use for making better decisions.
Infact, it is done in some forms in a few cases. Some call it Guiding-Principles. Some call it MVPs. I call it “Design Philosophy”.

When to Emit What O11y Signal?

Thu, 04 Sep 2025 00:00:00 +0000

The intention of this page is to put together the Observability Signal Guidelines which will provide the required visibility into the systems without hurting the cost aspect of the solution.

Three basic observability signals that any application emits are:

Metrics,
Traces and
Logs

The general question is - When to emit what signal?

The answer lies in the intent behind the signal being emitted. What do you intend to measure with the Observability signal that you are emitting?

Enhancing Observability with OTel Custom Processors

Sun, 27 Apr 2025 00:00:00 +0000

Observability is crucial for modern distributed systems, enabling engineers to monitor, debug, and optimize their applications effectively. OpenTelemetry (Otel) has emerged as a comprehensive, vendor-neutral observability framework for collecting, processing, and exporting telemetry data such as traces, metrics, and logs.

This blog post will explore how custom processors in OpenTelemetry can significantly enhance your observability strategy, making it highly customizable and powerful.

The repo link where I have implemented a very simple Otel-Custom-Processor.
https://github.com/AkshayD110/otel-custom-processor/tree/master

Memory management : Java containers on K8s

Sat, 08 Feb 2025 00:00:00 +0000

This page documents a few aspects of memory management on Java containers on K8s clusters.

For java containers, memory management on K8s have various factors:

Xmx and Xms limits managed by java
Request/limit values for the container
HPA policies used for scaling the number of pods

Misconfigurations / misunderstanding of any of these parameters leads to OOMs of java containers on K8s clusters.

Memory management on java containers:

-XX:+UseContainerSupport is enabled by default form java 10+

Observability - Metrics Madness

Tue, 07 Jan 2025 00:00:00 +0000

There isn’t a single book or article on observability (O11y) where there isn’t a mention of MELT. (Metrics, Events, Traces, Logs)
While these four are the building blocks of telemetry data in Observability, all the four components haven’t evolved at the same rate.

In this write-up, I delve into metrics in observability (mainly custom metrics) and argue how the overuse of metrics is turning into madness and directly impacting the cost of observability.