Akshay Deshpande

Metrics Types - When to use what ?

Thu, 30 Apr 2026 00:00:00 +0000

This writeup talks about different metric type that an application can emit as telemetry. It intends to cover the case of when to use which metric types and the usecase for them.

Metrics is one of the oldest forms of telemetry. There are many APM solutions, whose billing model runs on the number of metrics you send to them. But it is often not clear to app devs on what kind of metrics to emit from applications. Infact, the types of metrics can be confusing sometimes.

Prometheus: Static/Dynamic scraping on EKS

Thu, 20 Nov 2025 00:00:00 +0000

This note is a mental model for how Prometheus discovers and scrapes metrics in Kubernetes.

The lens I want to keep throughout is:

Where will the scrape config file sit?
(Prometheus repo vs application repo)
In which namespace will the serviceMonitor sit?
(and how Prometheus finds it)

At a high level there are two ways to tell Prometheus about a /metrics endpoint:

Static via in the Prometheus config file.
Dynamic via (CRD from Prometheus Operator) with label‑based discovery.

Leader/Follower relationship with Primary/Replicas

Mon, 10 Nov 2025 00:00:00 +0000

In most of distributed datastore systems, there are a lot of techical terms to describe the behavior of the system. While these terms, like, “Leader”, “Follower”, “Replication”, “Consistency”, etc., are widely used and helpful, what I feel missing are the details about internal relationship between these terms.
Analogically, while the map of the field is great, it is also important to understand how the soil, water, and sunlight interact to help the plants grow.

Encoding: From the POV of Dataflow paths

Wed, 22 Oct 2025 00:00:00 +0000

When studying Chapter 4 of Designing Data-Intensive Applications (Encoding and Evolution), I quickly encounters a level of granularity that seems mechanical: binary formats, schema evolution, and serialization techniques. Yet behind this technical scaffolding lies something conceptually deeper. Encoding is not merely a process of serialization; it is the very grammar through which distributed systems express and interpret meaning. It is the act that allows a system’s internal thoughts — the data in memory — to be externalized into a communicable form. Without it, a database, an API, or a Kafka stream would be nothing but incomprehensible noise.

Kubecon India : 2025

Thu, 25 Sep 2025 00:00:00 +0000

I attended Kubecon India 2025 held in Hyderabad this year. I mainly focused and attended talks related to Observability and Scalable designs.

Now that the sessions are uploaded to Youtube, linking the ones which I really enjoyed.

Observability at Scale With Monitoring as Code: Grafana, Prometheus, & Tempo - Vipin GopalaKrishnapillai & Saiabhinay Bommakanti, Amway Global - link
Predictable auto scaling with keda - link
Observability - tenant centric metrics - link
Building observability platform for Edge compute nodes - link

Design Philosophy: Observability

Sun, 14 Sep 2025 00:00:00 +0000

I enjoy philosophy. Stoic philosophy in particular.
Philosophy, I think, helps us revalidate our purpose. It acts as a yard stick and makes sure that we are not moving away from our First-Principles.

Applying the same to Software Engineering, in my opinion, every team should have a “Design Philosophy”. What is that one yard stick which teams can use for making better decisions.
Infact, it is done in some forms in a few cases. Some call it Guiding-Principles. Some call it MVPs. I call it “Design Philosophy”.

When to Emit What O11y Signal?

Thu, 04 Sep 2025 00:00:00 +0000

The intention of this page is to put together the Observability Signal Guidelines which will provide the required visibility into the systems without hurting the cost aspect of the solution.

Three basic observability signals that any application emits are:

Metrics,
Traces and
Logs

The general question is - When to emit what signal?

The answer lies in the intent behind the signal being emitted. What do you intend to measure with the Observability signal that you are emitting?

Networking and Protocols: 101 Notes

Thu, 19 Jun 2025 00:00:00 +0000

I recently did a brush up course on Networking/Protocols 101’s. Making my notes public.

Networking Basics / Protocols:

There are 7 layers of communications as defined in OSI (Open systems Interconnection) model

Enhancing Observability with OTel Custom Processors

Sun, 27 Apr 2025 00:00:00 +0000

Observability is crucial for modern distributed systems, enabling engineers to monitor, debug, and optimize their applications effectively. OpenTelemetry (Otel) has emerged as a comprehensive, vendor-neutral observability framework for collecting, processing, and exporting telemetry data such as traces, metrics, and logs.

This blog post will explore how custom processors in OpenTelemetry can significantly enhance your observability strategy, making it highly customizable and powerful.

The repo link where I have implemented a very simple Otel-Custom-Processor.
https://github.com/AkshayD110/otel-custom-processor/tree/master

Memory management : Java containers on K8s

Sat, 08 Feb 2025 00:00:00 +0000

This page documents a few aspects of memory management on Java containers on K8s clusters.

For java containers, memory management on K8s have various factors:

Xmx and Xms limits managed by java
Request/limit values for the container
HPA policies used for scaling the number of pods

Misconfigurations / misunderstanding of any of these parameters leads to OOMs of java containers on K8s clusters.

Memory management on java containers:

-XX:+UseContainerSupport is enabled by default form java 10+

Observability - Metrics Madness

Tue, 07 Jan 2025 00:00:00 +0000

There isn’t a single book or article on observability (O11y) where there isn’t a mention of MELT. (Metrics, Events, Traces, Logs)
While these four are the building blocks of telemetry data in Observability, all the four components haven’t evolved at the same rate.

In this write-up, I delve into metrics in observability (mainly custom metrics) and argue how the overuse of metrics is turning into madness and directly impacting the cost of observability.

Bookshelf

Wed, 01 Jan 2025 00:00:00 +0000

Books that have shaped how I think about technology, systems, philosophy, and life.

The Fountainhead

Ayn Rand

A story about integrity, individualism, and refusing to compromise on your vision. “Never ask people about your work. Don’t you know what you want?”

Unposted Letter

Mahatria Ra

Life progresses in a cycle of choice-consequence-choice. “Between the hated and the hater, it is always the hater who suffers more.” Put your peace above everything.

Talks

Wed, 01 Jan 2025 00:00:00 +0000

Talks and presentations I have given at conferences, meetups, and internal events. Mostly around observability, performance engineering, and distributed systems.

Talks

KubeCon India 2025

December 2025

Attended and participated in discussions around observability at scale and monitoring as code with Grafana, Prometheus, and Tempo.

More talks coming soon. If you’d like me to speak at your event, reach out at akshayd110@gmail.com

Message delivery in Distributed Systems

Sun, 17 Nov 2024 00:00:00 +0000

In distributed systems, the principle of message passing between nodes is a core concept. But this leads to an inevitable question: How can we ensure that a message was successfully delivered to its destination?

To address this, there are three types of delivery semantics commonly employed:

• At Most Once

• At Least Once

• Exactly Once

Each of these offers different guarantees and trade-offs when it comes to message delivery. Let’s break down each one:

Personal "FinOps" with Ledger cli

Mon, 28 Oct 2024 00:00:00 +0000

This post is a geek-out journey this festive season on finding the right tool for my personal finance management.

Where it all started:

Recently, I spoke at the Smarsh Tech Summit on “Cost as an Architectural Pillar,” where I emphasized the importance of considering cost as a first-class citizen in the software development cycle.

However, on a similar note, when I was looking through my personal finances later, I wasn’t very happy when I had to apply the same principle.
I was using one of the apps for finance management, and it was all over the place. Since it is festive time off at work, I started looking around for the best way to fix this and track my personal finance the right way.

Knowledge management with Obsidian

Tue, 17 Sep 2024 00:00:00 +0000

This is a brain dump on how taking notes and Obsidian as a tool has helped me.

Knowledge management?

As one progresses further into career, Knowledge management becomes as equally important as Finance management. Knowledge accumulation is a non-linear trajectory. Majority of the times it is compounding in nature. If one doesn’t organise it, you are always at the mercy of “I had solved this once before but don’t remember how”.

Bloom Filter and Search Optimisation

Sun, 23 Jun 2024 00:00:00 +0000

This writeup is an outcome of a side quest while geeking out on System Design.
In the book “Designing Data-Intensive Applications,” Bloom Filters are briefly mentioned in the context of datastores, highlighting their significance in preventing database slowness caused by lookup for nonexistent items.

Below are a curious set of questions on the topic on Bloom Filters and how it works.

What is the use case for a Bloom filter?

Imagine you are maintaining a Datastore which has millions of records. You want to search for an item form the Datastore, while you are not sure that the item exists in the first place.

Weekly Bullet #42 – Summary for the week

Sat, 23 Mar 2024 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

The cost crisis in the observability space is a real problem. Here is an article that describes the issue: - link here
How many conferences are too many? Here is an exhaustive list of all the popular talks on Kubernetes from 2023: - link here
OpenTelemetry is the industry standard in observability. Here is a list of anti-patterns with observability to avoid:- link here

Kafka - an efficient transient messaging system

Fri, 01 Mar 2024 00:00:00 +0000

Over the past few years, I have worked on different multi-cluster distributed datastores and messaging systems like - ElasticSearch, MongoDB, Kafka etc.

From the Platform Engineering/SRE perspective, I have seen multiple incidents with different distributed datastores/messaging systems. Typical ones being :

uneven node densities (ElasticSearch - how are you creating shards?)
client node issues (client/router saturation is a real thing. And they need to be HA)
replicas falling being masters, (Mongo - I see you)

Weekly Bullet #41 – Summary for the week

Thu, 25 Jan 2024 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

System Design - Designing a Ticket Booking Site Like Ticketmaster is the most common system design question - link
UberEngineering blog on Anomaly detection and alerting system - link
P99 CONF 2023 | Always-on Profiling of All Linux Threads by Tanel Poder - YouTube link
On choosing Golang as a programming language at American Express- link

[Kubernetes]: CPU and Memory Request/Limits for Pods

Sun, 14 Jan 2024 00:00:00 +0000

In this write up, we will try and explore how to make the most out of the resources in K8s cluster for the Pods on them.

Resource Types:

When it comes to resources on Kubernetes cluster, they can be fairly divided in to two categories:

compressible:
- If the usage of this resource for an application goes beyond the max, it can be throttled without directly killing the application/process.
- example : cpu - if a container consumes too much of compressible resource, they are throttled

[DDIA Book] : Data Models and Query Languages

Tue, 10 Oct 2023 00:00:00 +0000

[Self Notes and Review]:

This is a second writeup in the series of reading DDIA book and publishing my notes from the book.
The first one can be found here

This particular article is from the second chapter of the book. Again, these are just my self notes/extracts and treat this more like an overview/summary. Best way is to read the book in itself.

This chapter dwells in to the details of: the format in which we write data to databases and mechanism by which we read it back.

Weekly Bullet #40 – Summary for the week

Mon, 18 Sep 2023 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

The shortest and comprehensive System Design Template for any new service - link here
Kafka is one of the most efficiently built transient datastore. This article explains the compute and storage layers of kafka – link here
Consistent Hashing has helped solve Distributed System with:
- even shard distribution across nodes in cluster

Weekly Bullet #39 – Summary for the week

Mon, 24 Jul 2023 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

AI-Powered Search and Chat for AWS Docs. The best way of consuming AWS docs. – link here
A Framework for Thinking About Systems Change - link here
BookWyrm - Yet another attempt on building a social media based on book. link here
- git repo for the same - bookwyrm
- Built by Mouse Reeve - https://github.com/mouse-reeve

[DDIA Book]: Reliable, Scalable and Maintainable Application

Wed, 05 Jul 2023 00:00:00 +0000

[Self Notes and Review]:

This is a new series of publications where I am publishing my self notes/extracts from reading the very famous book - DDIA (Designing Data-Intensive Applications) by Martin Kleppmann.

This particular article is from the first chapter of the book. Again, these are just my self notes/extracts and treat this more like an overview/summary. Best way is to read the book in itself.

Side note: I am a terribly slow and repetitive reader. The update between chapters might take weeks.

Weekly Bullet #38 – Summary for the week

Sun, 04 Jun 2023 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

Don’t turn that Swap off yet. In defense of swap - common misconceptions. - link here
Why histogram and how are they useful. link here
- If you have used any of the monitoring or APM tools, you would have come across Histogram form of metric being emitter. This writeup give a brief on why Histogram.
S3 isn’t getting cheaper - link here

Weekly Bullet #37 – Summary for the week

Mon, 17 Apr 2023 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

A new book released on the hot eBPF - “Learning eBPF” by Liz Rice. This a summary form and a quick introduction to eBPF capabilities when compared to “BPF Performance tools” by Brendan Gregg.

To understand latency in detail - “Everything You Know About Latency Is Wrong” - link here

Weekly Bullet #36 – Summary for the week

Sat, 04 Feb 2023 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

[Video-57mins] : What is Continuous Profiling in Performance monitoring and What is Pyroscope - with Ryan Perry - link
Go 1.20 is here(link). A thread on all the changes - here
“What’s the best lecture series you’ve seen?” - Thread link
Some great side project idea on the thread - here. My fav is PlainTextSports

Weekly Bullet #35 – Summary for the week

Fri, 02 Dec 2022 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

It is December and Advent of code is here. What is Advent of code ? - link here. An old podcast on Spotify’s Engineering team geeking out every December on AOC - link here
[A talk - 31mins ] - Concurrency is not Parallelism - “Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.” - YouTube

[Memory-metrics]: Linux /proc interface

Mon, 22 Aug 2022 00:00:00 +0000

This writeup is more of a demo to showcase the power of “proc” (process information pseudo-filesystem) interface in linux to get the memory details of process, and also a quick brief on the power of “proc interface”.

In the current trend of building abstraction over abstractions in software/tooling, very few tend to care about the source of truth of a metrics. There are various APM / Monitoring tools to get the memory details of a process for a linux system, but when the need arises, I believe, one must know the ways of going closer to the source of truth on a linux system and verify things.

Weekly Bullet #34 – Summary for the week

Fri, 19 Aug 2022 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

[Book-Recommendations]: Cloud-Native Observability with OpenTelemetry by Alex and Charity. (Side note: Half way through the book and learning a lot. )

Take away : Effective ways of adding metrics/traces in cloud native apps without marrying to any APM tools

A list of possible developer questions to consider asking a prospective employer. - github link
[Video]: Disk latency being directly proportional to vibrations (in other words, lets try shouting in data center!!) – 2min Video link
What is Redis explained - link here
Best Practices for Fixing your alerts in your service? (so that you can sleep well?) – link from newrelic here
The disproportionate influence of early tech decisions (debatable but interesting!) - link here

Non-Technical :

Second-order thinking: How to NOT create new problems while solving existing problems. - Link
[90mins] Podcast recommendation – The Knowledge Project : Insights for making better Decision – Spotify link
What’s the strangest thing you ever found in a book ? (heartwarming!)– link
10 mental concepts for thinking better - Twitter thread
A quote that struck the right chords:

“So much advantage in life comes from being willing to look foolish in the short term.”

Docker: A list of most frequently used commands

Thu, 07 Apr 2022 00:00:00 +0000

This writeup is a dump of my study notes on most frequently used docker commands for reference. This is just a self reference page and will get updated on the go

To run a container from an image

docker run <image name>

To run a docker image with a specific tag. Example below of pulling redis image with tag4.0. You will get these tag details on the dockerhub page for the image

docker run redis:4.0

To run a docker image in detached mode

docker run -d <image_name>

To run a docker image and login to the container directly

docker run -it <image_name>

To list all the docker images

docker images

To pull a docker image from dockerhub but not run it.

docker pull <image_name>

To list all the docker containers

docker ps -a

To stop a docker container

docker stop <container_name>

To remove a docker container form the disk.
Note: This will remove the container permanently. It will not list anymore in docker ps -a. However, the image still exists. The exited/stopped containers do not consume any CPU or memory, but they still use the machine’s disk space.

docker rm <container_name>

To remove a docker image

docker rmi image

To execute a command in a running docker container

docker exec <container_name> <command>

To get the ip of a docker container

docker inspect <container_id/container_name> | grep IPAddress

To map the internal port of a docker container to a host port

docker run -p 80:5000 <image_name>

To get the logs of a container

docker logs <container_name>

To build a docker file

docker build . #from being in the dir which has Dockerfile

To map an external directory at the bootup to a docker container

docker run -v /myCustomdir:/defaultDir -u root imageName

Weekly Bullet #33 – Summary for the week

Thu, 17 Mar 2022 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

A few interesting and hidden features of Python - link
Below are the BCP tools that can be used for digging in to Performance analysis of memory parameters on a Linux machine. More in BPF Performance Tools book by Brendan Gregg.

Book : BPF Performance Tools

How does Database indexing work? - here
How to get the most out of your 1:1s - here
At around 8-10years of experience, career branches in to either Engineering management or Technical Staff Engineer. Here are a few stories of Staff Engineers and their journey.
[Self-endorsement]: A tiny 50lines of code tool for getting my highlighted quotes from my Fav books - here

Non-Technical :

[Highly recommended]: First-principles thinking is a competitive advantage because almost no one does it. More here
The importance of annual health checkup - Seema’s True story of battling cancer - here
I always thought about a social platform but based on Books. Booqsi is trying to do the exact same. It is still in early beta though. More details here
A website that removes things from images in seconds- Magic Eraser
What Got You Here Won’t Get You There: Book Summary - here
Podcast : Hugh Jackman - on Daily routine, intuition, meditation, decision making and more. link
A quote that the struck right chords:

“If you are not willing to take responsibility for your situation, you cannot use your situation as an excuse either.”

[Tiny tool]: Book Extract Reminders

Mon, 07 Feb 2022 00:00:00 +0000

There is no better pleasure than the Joy of solving your own problems.

This write up is not to show off coding skill(there is hardly any code in this tool), but to show the ease with which anyone can build tools to solve problems these days.

Problem statement:

How to retain the most out of the books we read? Maybe receiving daily reminders with extracts from the books?

I consume books mainly in the digital format(via kindle/calibre). I have a lot of highlights in these books which I want to be periodically reminded about. I felt, if I take out 6hours in reading a book, and completely forget all learnings in next 6months, then that’s not efficient.

Weekly Bullet #32 – Summary for the week

Mon, 17 Jan 2022 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

Memory leaks on client side - the forgotten side of web performance. Link
A list of helpful patterns/commands on "_sed_" command - link
A “Streaming availability” api to lookup which show/movie is available in which OTT in 60+countries. Something to explore for a fun weekend project - link
You remember the times, when there are 10 terminal sessions opened on your system and you wanted to know “at what time did I run that command” ? Ya, article addresses the same problem - link
PS: If you use zsh, there a few themes which come in-built with this feature.
Wordle puzzles are crazy popular in the last week or so. Here is a python project to solve the puzzles. link

Non-Technical :

Completing part-time master’s in CS while on a full time job! This article was so inspiring also reminds, how much time we waste in general. - link
Drop a raindrop anywhere in the world and watch where it ends up. A fun site - link
Rocket engines, they deal with a lot of heat right? How come they don’t melt? Detailed geeky explanation - here
An interesting Thread on Human psychology fact. link
[Image below]: “Stop focusing on the black lines behind you. Start focusing on all of the green lines before you.”

Weekly Bullet #31 – Summary for the week

Sun, 09 Jan 2022 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

CPU utilization is wrong - PS : idle waits are counted in the %CPU.
Python : Comprehensive Python Cheatsheet
Finished coding, but waiting for PR to be review/approved ? - The Pull Request Paradox
Best practices can slow your application down - Best Practices vs Required Practices - link
Profiling and analyzing performance of Python programs - link
Also, since it is Monday:

Weekly Bullet #30 – Summary for the week

Mon, 22 Nov 2021 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

P99 CONF (centered around low-latency, high-performance design) recordings are available here

Python f-strings can do more than you thought. Video here
Web3 is not a hype. An article on what is Web3 and decentralized internet here. Also, a podcast by Tim Ferris with Chris Dixon and Naval Ravikant – The Wonder of Web3.
Another one of those – “20 Things I’ve learned in my 20 years as a Software Engineer.”
Most helpful sed one liners - here
How WhatsApp scaled to 1 billion users with only 50 Engineers. - here
The “Advent of code” will be live on December 1st. Here is some history about it. Discussion thread.

Non-Technical :

Traveling(rather walking) without money! Although old(1998) event, still shows world is not that bad of a place. Link “MY PENNILESS JOURNEY”
There was this mega-thread on twitter for “one book that changed the way you see the world”. - The consolidated list from the thread on BooksChatter here.
Matthew McConaughey addressing University of Houston outgoing students – “5 Rules for the life” (Rule #1 is my fav)
Extract from a book:

“If you think about the biographies you read or the documentaries you watch about the greats in various fields, this same pattern of Addictive, Passionate behavior surfaces. Jazz saxophone great John Coltrane reportedly practiced so much that his lips would bleed.”

Weekly Bullet #29 – Summary for the week

Thu, 19 Aug 2021 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

eBPF Summit is live now - Recording of the Keynote and live summit here

Conference talk - USENIX LISA2021 Computing Performance: On the Horizon by Brendan Gregg - here [video - 41mins]
A project for visualizing codebase - here
Self healing systems - the real end goal of observability - here
Python3 - Reverse Engineering Tips - here

Non-Technical :

Great article on - “How to think: The skill you’ve never been taught” - here
An addictive trading game in your browser - Paper trade - here
People no longer trust each other. Why? And how can we fix it? An interactive guide to the game theory of trust - here
What tiny purchases have disproportionately improved your life? - Thread link
Extract from a book:

“But we had two choices,” I said. “Throw our hands up in frustration and do nothing, or figure out how to most effectively operate within the constraints required of us. We chose the latter.

[Performance] : Understanding CPU Time

Sat, 24 Jul 2021 00:00:00 +0000

As a Performance Engineer, time and again you will come across a situation where you want to profile CPU of a system. The reasons might be many; like, CPU usage being high, you want to trace a method to see its CPU cost or you suspect CPU times for a slow transaction.

You might use one of the various profilers out there to do this. (I use yourkit and Jprofiler). All these profilers report the CPU costs in terms of CPU Time, when you profile the CPU. This time is not the equivalent of your watch time.

Weekly Bullet #28 – Summary for the week

Sun, 11 Jul 2021 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

All recording from PyCon US 2021 are up on Youtube here. My fav is Keynote by Robert Erdmann about rebuilding 5 µm resolution picture of Rembrandt’s painting “The Night Watch” from 18th century with Python.

Rembrandt’s painting “The Night Watch” from 18th century

“Docker For The Absolute Beginner” course. This is offered free on kodekloud.com . The same course was taken by over 97,000 students on Udemy.
datefinder is an amazing python module for location date out of different date formats in a string. Here is a short video about the same.
Book recommendation - “BPF Performance Tools” - By Brendan Gregg.
BPF-based performance tools give you unprecedented visibility into systems and applications, so you can optimize performance, troubleshoot code, strengthen security, and reduce costs.

Elastic Search Best practices

Mon, 14 Jun 2021 00:00:00 +0000

These are the self-notes from managing 100+ node ES cluster, reading through various resources and a lot of production incidents due to unhealthy ES.

Memory

Always choose ES_HEAP_SIZE 50% of the total available memory. Sorting and aggregations both can be memory hungry, so enough heap space to accommodate these is required. This property is set inside the /etc/init.d/elasticsearch file.
A machine with 64 GB of RAM is ideal; however, 32 GB and 16 GB machines are also common. Less than 8 GB tends to be counterproductive (you end up needing smaller machines), and greater than 64 GB has problems in pointer compression.

CPU

Choose a modern processor with multiple cores. If you need to choose between faster CPUs or more cores, choose more cores. The extra concurrency that multiple cores offer will far outweigh a slightly faster clock speed. The number of threads is dependent on the number of cores. The more cores you have, the more threads you get for indexing, searching, merging, bulk, or other operations.

Disks

If you can afford SSDs, they are far superior to any spinning media. SSD-backed nodes see boosts in both querying and indexing performance.
Avoid network-attached storage (NAS) to store data.

Network

The faster the network you have, the more performance you will get in a distributed system. Low latency helps to ensure that nodes communicate easily, while a high bandwidth helps in shard movement and recovery.
Avoid clusters that span multiple data centers even if the data centers are collocated in close proximity. Definitely avoid clusters that span large geographic distances.

General consideration

It is better to prefer medium-to-large boxes. Avoid small machines because you don’t want to manage a cluster with a thousand nodes, and the overhead of simply running Elasticsearch is more apparent on such small boxes.
Always use a Java version greater than JDK1.7 Update 55 from Oracle and avoid using Open JDK.
A master node does not require much resources. In a cluster with 2 Terabytes of data having 100s of indexes, 2 GB of RAM, 1 Core CPU, and 10 GB of disk space is good enough for the master nodes. In the same scenario, the client nodes with 8 GB of RAM each and 2 Core CPUs is a very good configuration to handle millions of requests. The configuration of data nodes is completely dependent on the speed of indexing, the type of queries, and aggregations. However, they usually need very high configurations such as 64 GB of RAM and 8 Core CPUs.

Some other important configuration changes

[Performance] : What does CPU% usage tell us ?

Fri, 28 May 2021 00:00:00 +0000

When you come across a system which is misbehaving, majority of the times the first metrics that we look at is CPU usage. But do we really understand what CPU usage of a system tells us ? In this article let us try and understand what X % usage of a system really means.

One of the easy ways to check on CPU is “top” command.

The “%Cpu(s)” metrics seen above is a combination of different components.

Weekly Bullet #27 – Summary for the week

Mon, 17 May 2021 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

Different states of Java Threads and their transitions. - Link
A quick look into Sorting in python - RealPython site link (3mins)
DevOps in one picture:

Source: https://aws.amazon.com/devops/what-is-devops/

A cheat sheet to “When to use which collection in java” - here

source: http://www.sergiy.ca

A great talk on internals of List and Tuple in Python - YouTube (28mins)

Non-Technical :

A crisp explanation on Manager vs Director vs VP - link

Weekly Bullet #26 – Summary for the week

Mon, 10 May 2021 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

“Performance checklist for SREs” - By Brendan Gregg at SREcon16 . YouTube link (1hr)
Resource list for Beginner to Pro in Python. Link
Navigation in IntelliJ IDEA - this could save so much time once all short cuts are know. YouTube link (8mins)
Monitoring SRE’s Golden Signals - The metrics that matter and the ones we absolutely need to monitor. Link
All The Important Features and Changes in Python 3.10. Link
“What makes a Great Software Engineer?” - An IEEE paper on non-technical qualities of a great Software Engineer. Link

Non-Technical :

[Highly Recommended] : Henry Rollins: The One Decision that Changed My Life Forever | Big Think - YouTube link (7mins)
Now that most of use working from home, here is mynoise.net for creating Quiet Animated Atmospheres. How to use - here
A great site for some fun riddles - here
Extract from a book :

“A professional is someone who may not have all the answers, but thoroughly studies their craft and seeks to hone their skills. A professional will freely admit when they don’t know the answer, but you can count on a professional to find it.”

Weekly Bullet #25 – Summary for the week

Wed, 17 Mar 2021 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

Amazon S3 on it’s 15th Birthday – It is Still Day 1 after 5,475 Days & 100 Trillion Objects. An article here.

A detailed Performance comparison of different programming languages / command-lines. Link here. (If you can’t read full article, go through the conclusion for insight)
The Amazon VP & CTO, Werner Vogels sits with Tom Killalea to discuss designing for evolution at scale. Article here.
ShortcutFoo is a site for spaced repetition of helpful shortcuts across tech stacks. Check it here.

Non-Technical :

Flamshot, an amazing multi-functional screenshot capturing tool. Check it here . Download link.
(Highly Recommended) : The context of “Why’s!” by Richard Feynman. Youtube link [Length - 7min]
Tim Ferriss podcast with Jordan Peterson(Canadian professor of psychology) as a guest. You can definitely learn new things here - link. [Youtube. Length - 1hr 20mins]
Extract from a book:

“One lesson I’ve learned is that if the job I do were easy, I wouldn’t derive so much satisfaction from it. The thrill of winning is in direct proportion to the effort I put in before. I also know, from long experience, that if you make an effort in training when you don’t especially feel like making it, the payoff is that you will win games when you are not feeling your best. That is how you win championships, that is what separates the great player from the merely good player. The difference lies in how well you’ve prepared.”

Weekly Bullet #24 – Summary for the week

Wed, 17 Feb 2021 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

An overview of iftop - a great network traffic visual tool. - here , also man page here
Rust is becoming one of the most loved languages. Here is an Illustrated Note about WTF is Rust - link
“How They SRE” - best practices, tools, techniques, and culture of SRE adopted by the leading technology or tech-savvy organizations.- link
A single stop to find all upcoming Tech conferences in 2021 - link
A wiki on Unix Toolbox with all the commands and tasks useful for daily dive in to linux world. - link
“Python Tricks I cannot live without” - link
I am sure most of you follow HackerNews. Here is a great tool built using FlameGraphs to navigate through big threads on HN. - Link1 , Link2

Non-Technical :

A cool site where you can select the part of the body and find the relevant stretches and exercises here
An extract from something I am reading:

“Almost universally, the kind of performance we give on social media is positive. It’s more “Let me tell you how well things are going. Look how great I am.” It’s rarely the truth: “I’m scared. I’m struggling. I don’t know.”

[Performance] : Using iperf3 tool for Network throughput test

Sun, 31 Jan 2021 00:00:00 +0000

In this world of Microservices and the distributed systems, a single request (generally) hops through multiple servers before being served. More often than not, these hops are also across the Network cards making the Network Performance the source of slowness in the application.
These parameters makes the need to measure Network performance between servers/systems more critical for benchmarking or debugging.

Iperf3 is one of the open source tools which can be used for network throughput measurement. Below are some of its features.

[Performance] : Java Thread Dumps - Part2

Wed, 30 Dec 2020 00:00:00 +0000

In the previous article about Java Thread Dumps (link here) we looked in to a few basics on Thread dumps(When to take?, How to take?, Sneak peaks? etc.)

In this write up, I wanted to mention a few tools which can ease the process of collecting and analyzing thread dumps.

Collecting multiple thread dumps:

I prefer command-line over any APM tools for taking thread dumps. The best way for analyzing threads is to collect a few thread dumps (5 to 10) and look through the transition in the state of threads.

Weekly Bullet #23 – Summary for the week

Tue, 29 Dec 2020 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

BPF(Berkeley Packet Filter) has come a long way from just being a packet capture tool to advance Performance analysis tool (EBPF - Extended Berkeley Packet Filter). Here (link) is an introduction to EBPF. Also here (link) is a talk on how BPF is used at Netflix.

“Minimal safe Bash script template” - link . Because there is no such thing as “knowing enough of bash!”
Kelsey Hightower is an inspiration. A writeup on how he made it from McDonald’s to Google (link). [HIGHLY RECOMMENDED] –> : A talk he gave about his journey a few years back here (link)
[That time of the year!] : “Best talks of 2020” – link
[Late news!] If you didn’t hear it already, Github has Dark mode now. - link

Non-Technical :

[Another one] “Ask HN: What book changed your life in 2020?” - some great recommendations here - link . Personally for me, “Sapiens” widened my horizon about evolution of Human Beings.
“100 Tips for better life.” - link - I don’t agree with all of them, but most of these are thought provoking.
An extract from the book that I am reading.

At the core of all anger is a need that is not being fulfilled.

Weekly Bullet #22 – Summary for the week

Mon, 30 Nov 2020 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

[Talk-Velocity 2017] : Performance Analysis Superpowers with Linux eBPF (44mins)- link
Popular Java Podcasts to follow in 2020 - link
Since we are taking about Podcasts, I have also heard good things about Barcode and ACM-ByteCast is amazing!
Illustration: Much that we have gotten wrong about SRE - link
A list of popular java libraries. - link
The second edition of “System Performance: Enterprise and Cloud” - by Brendan Gregg releasing on 2nd December. - link . This is “the best” reference guide for Performance Engineering.

Non-Technical :

“Library of Scroll” - Here is a site with one great article every Monday. Since it is just one, generally I find them very good. - link
Great site with short explanations of over 24 cognitive biases. Co-authored by Gabriel Weinberg who is the CEO of DuckDuckGo. - link
Not sure why I liked this, but this “57 Years Apart - A Boy And a Man Talk About Life” short video was quite gripping. - link
Soft skills for Software Engineers. Short thread. - link
Hand picked remote jobs from “Hacker News Who is hiring” November - link
Extract from a book:

“Respect an old tradition path as it is well tested, but also be open to the new modern way of things as they open up your mind.”

Thinking in-terms of Performance

Fri, 20 Nov 2020 00:00:00 +0000

A few short thoughts / ideas wrt of Performance centric product.

In this world of infinite scaling of computes, pay close attention to common choke points. Like DB, storage(s) etc, which are shared by all the computes.
Majority of the reads and writes have to happen in Bulk operations and NOT as single read/writes. Specially when there are 100’s-1000’s of reads/writes/deletes on storage(s).
Threads. Pay close attention to which part of the entire flow is multi-threaded. Sometimes, only a small part of the flow is multi-threaded, but entire application is called multi-threaded, which is wrong.

Weekly Bullet #21 – Summary for the week

Mon, 26 Oct 2020 00:00:00 +0000

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical :

How to become a consultant ? Some good references and advices here. - link
Ever wondered How the prices vary on Amazon? Here is a classic example of algorithmically priced products on Amazon - link
If you are using O’Reilly (which I believe is the best technical content platform), you should check out O’Reilly Answers. For all your queries, O’Reilly looks through heap of books, video and conferences and gives you answers. - link (Note: You will need subscription)
In Firefox version 81, an experimental event delay tracker has been added. Details in the “Performance Tools” section of below article (old article dated August 31st). - link . Also example profile - link
An extensive collection of Bash pitfalls - link

Non-Technical :

As “Arguments” are a part of corporate jobs, here is a Beginner’s Guide to Arguing Constructively. - link
Expiring vs Permanent skills. Many would agree that these skills are more important than absolute technical skills. - link
Nokia is going to build a mobile network on Moon - link
If you are into living in small spaces and minimalism, you will love this youtube channel, “Never too small”. link . Also, my recent favorite - link
[Repost, because why not!] - This never gets old. Richard Feynman’s - “Names Don’t Constitute Knowledge” principle (2mins) - link
An extract from a book that I am reading:

Fooling with books so you can sound smart or have an intimidating library is like tending a garden to impress your neighbors.

[Performance] : Java Thread Dumps - Part1

Thu, 22 Oct 2020 00:00:00 +0000

This is first of a two parts article which talks about:

What are thread dumps?
When to take thread dumps ?
How to take thread dumps ?
What is inside a thread dumps ?
What to look for in a thread dump?

Majority of the systems today are mutlicore and hyper-threaded. Threading at the software level allows us to take advantage of a system’s mutlicores to achieve the desired pace and efficiency of the application operations. Along with pace and efficiency, multi-threading brings its own set of problems w.r.t thread contentions, thread racing, high CPU usage etc. In this write up we will see how to debug these problems by taking thread dumps on java applications.

[Performance] : Flame Graphs

Sun, 26 Apr 2020 00:00:00 +0000

In the previous article we explored the basic capabilities of linux Perf_tool.
In this write-up I am trying to extend these capabilities and show how to generate and read Flame Graphs for analyzing the profiles generated with Perf_tool.

How to generate Flame Graphs ?

To start with, we will need perf_tools linux profiler to capture the profile first. Follow the steps under “How to setup perf tool?” in the previous article.
Now if you collect a profile of the CPU using perf_tool setup in the above step, there is a possibility that you might see a lot of symbol link values in the place of Function names.

[Performance] : Profiling with linux Perf command-line tool

Sun, 15 Mar 2020 00:00:00 +0000

Most of the Performance Engineers use some sort of profiling tools like Yourkit, Jprofiler or some APM tools like Newrelic, Datadog, Appdynmics etc. Although these tools are easy to use out of the box and help with Observability, they don’t give a complete picture of a Performance problem at occasions.
This is where perf Linux profiler comes in handy.

This write up is an attempt to explain :
- What is perf Linux profiler ?
- How to set it up ?
- What are its capabilities ?

[Performance] : Java's built in diagnostic tool - Jstat

Wed, 22 Jan 2020 00:00:00 +0000

When it comes to Performance Monitoring and analysis, we tend to think of full fledged license tools like Dynatrace, Newrelic, Appdynamics, Yourkit etc. However, if it is a java application which is under diagnosis, java’s built in tools are a good place to start.

Java comes with a set of built-in diagnostic tools like - Jconsole, jcmd, jstat, jmap, jstack, jvisualvm, jfr and many more. Each of them help in tackling a kind of problem. For the scope of this article, lets look in to how jstat is useful as diagnostic tools.

Weekly Bullet #20 – Summary for the week

Fri, 10 Jan 2020 00:00:00 +0000

Hi All,

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical:

107+ Coding Interview Problems with Details Solutions. - link
“How do I choose the right resource to learn CS fundamentals?” - Some great resource links in the comments - link
“Paper Digest” - a site for newly published research papers. Has variety of topic to subscribe for. - link
“List of favorite books on CS concepts/theory” - link
“Project lovable” - a site for free scientific programming problems. - link
A collection of interactive tutorials, guides and quizzes about maths, algorithms, performance, and programming languages. - link

Non-Technical :

A twitter thread from Sam Altman (chairman of Y combinator) on “How to be successful at your career” - link
The Levels.fyi annual report for software engineering compensation - “Highest paying companies of 2019” - link
[Recommended] “Be an easy employee to manage’ - a great thread with insightful comments. - link
“Yes, It’s All Your Fault: Active vs. Passive Mindsets” - Great read - link
An extract from a book I am reading :

“There is difference between losing and being beaten. Being beaten means they are better than you. They are faster, stronger and more talented. … But losing means you lost focus. It means you didn’t concentrate on what was essential. To operate at your highest level of contribution requires that you deliberately tune in to what is important in the Here and Now.”

Weekly Bullet #19 – Summary for the week

Fri, 13 Dec 2019 00:00:00 +0000

Hi All !

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful.

Technical:

Most popular and highly recommended technical books. The books are segregated in to variety of topics. - link
htop explained in depth - link
A person who got 7 offers from companies including Google, sharing his experience on how he did it. - link
Roadmaps for becoming a Frontend dev, Backend dev or a DevOps expert. - link
Interesting visualization of Bubble-sort with Hungarian folk dance. - link
How Shopify successfully merges the work of 1000+ developers everyday. - link. Also, there is a discussion with the author of the post here

Non-Technical:

For the ones who don’t have time for books, here is a way to get a Page per day of classic books delivered to your inbox. - link
Although Feynman’s approach for learning is mentioned here so many times, here is another writing on “The Secret algorithm behind learning” - link
A tediously accurate model of the solar system which is a unique educational experience that depicts the scale of the universe. - link
How to write better emails (double recommended) - link
A short quote from a book :

“I’m always suspicious of the ones who say everything’s going well. If you think that things are going well, then it’s usually some kind of arrogance. If it’s too easy for you, you just relax. You don’t make a real effort, and therefore you never find out what it is to be fully human.”

[Monitoring] - Paging and Swapping in Memory

Mon, 09 Dec 2019 00:00:00 +0000

As a continuation after understanding Virtual Memory in the previous article, this article tries to explain theways to monitor the same.

Memory can be looked at with two perspectives: Utilization and Saturation.
Utilization tells the memory usage. Checking free/used memory reflects the Utilization.
Saturation tells if the memory is used at its full capacity and how the system is using Virtual memory to deal with memory crunch.
In other words, if demands for memory exceed the amount of main memory, main memory becomes saturated. The operating system may then free memory by employing paging, swapping, and, on Linux, the OOM killer. Any of these activities is an indicator of main memory saturation.
Also, it is important to understand that Paging and Swapping are two different things. More details about the same in the previous article.

Weekly Bullet #18 – Summary for the week

Fri, 08 Nov 2019 00:00:00 +0000

Hi All!

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

A great post on different Linux tracing systems, which can used for Performance debugging, and how they fit together by Julia Evans. - link

Author : Julia Evans

A talk from “Strange Loop Conference 2018” on “A Practical Look at Performance Theory” by Kavya Joshi - link
Becoming a manager is not the only way up the ladder. If you enjoy getting you hands dirty with technology or if you are not a people person, Technical Leadership path is also an option. More thoughts in this great article - link
A git resource repository with 30Day code challenge and resources for Algorithms, Data Structures implementation and more. - link
List of softwares (for CI/CD, Code quality, Security) that has free tiers for developers. - link

Non-Technical:

Over 1300 Free Online courses from Top Universities on wide variety of subjects like History, Architecture, Economics etc. - link
On the importance of Morning routine by American author Daniel Pink. - link
Podcast recommendation - A podcast from Tim Ferriss with Ben Horowitz. Great talk around leadership in technical fields. - link
An extract from the book - “A guide to the good life”

" You are living in what to your ancestors would have been a dream world. You take for granted things that your ancestors had to live without. "

[Understanding] : Virtual Memory

Mon, 04 Nov 2019 00:00:00 +0000

As a Performance Engineer you will come across Virtual memory very often specially when monitoring or debugging Memory issues. Virtual memory along with below mentioned terminologies are used very loosely across the industry.
- Page, Page frame, Page fault, Minor/Major fault, Paging, Swapping etc.

This article is an attempt to understand Virtual memory in detail theoretically, in the context of Computer Architecture.

What is virtual memory?

Virtual memory is not the real memory. It is an abstraction layer provided to each process. It is meant to simplify the software development, leaving the physical memory placement to operating system.
To put in very simple terms, the purpose of Virtual memory is to use the hard disk as an extension of RAM, thus increasing the available address space a process can use. Using Virtual memory, system can address more memory than it actually has, and it uses the hard drive to hold the excess. This area on the hard drive is called a page file, because it holds chunks of main memory on the hard drive.

[Understanding] : How to read a G1GC log file.

Fri, 11 Oct 2019 00:00:00 +0000

As a Performance Engineer, time and again you will need to look in to GC logs to see how jvm is handling garbage collection.
With G1GC being the default gc for java versions 9 & above, one needs to know what the G1GC log actually reads like.

To get an understanding of G1GC, here is an in-depth material on it from Oracle tutorials. - link

To begin with, there are 50+ jvm parameters for selecting different GC algorithms and customizing them as per requirement.
Below link has cheat sheet for all the jvm parameters that you can select from. - PDF

Shell Scripting - Functions [Part2]

Sun, 06 Oct 2019 00:00:00 +0000

This post is a followup on the first article - basics of Shell scripting [here - note: increasing the scope beyond Performance Engineers only].

In this write up we look at how to modularize a shell script using Functions and how to create a set of useful functions -> convert them in to library -> use them across scripts.

Functions:

It is a good practice to write shell scripts as functions rather than stand alone scripts so that they can be easily incorporated in to other scripts without incurring the overhead of system calls. While there is no import feature like in Python, there are capabilities of Sourcing files is shell scripts.
But first, lets look at ways of writing functions and invoking them.

Weekly Bullet #17 – Summary for the week

Sat, 21 Sep 2019 00:00:00 +0000

Hi All!

To make these posts more Quality oriented, I am trying to reduce the frequency of posts from weekly to fortnightly (based on content). Simply put - I will not write if I don’t have anything that will add value to you.

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

A talk from “Strange Loop-2019 conference” on “Performance Matters” by Emery Berger. Covers how a small tiny thing can impact performance, importance of randomizing tests(Stabilizer tool) etc.
All lectures from “MIT 6.006 Introduction to Algorithms, Fall 2011” class.
A study guide to tell you - What to study and Why to study, along with resource recommendation. “TeachYourSelfCS.”

Weekly Bullet #16 – Summary for the week

Sun, 01 Sep 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

A large number of people involved in Tech industry do not know Coding. A great small post on advice to the same group - “The Surprising Number Of Programmers Who Can’t Program”
Another Git repo for a wide set of Computer Science Resources - “ComputerScienceResources.”
Book recommendation : “Web Performance basics”. It talks about the basic of Web waterfall charts, Profiling charts, CPU & Memory profiling for web etc.

Weekly Bullet #15 - Summary for the week

Fri, 02 Aug 2019 00:00:00 +0000

Hi All!

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

What if I told you that CPU % that you always monitor is Wrong! Did you know that requests stalled (waiting) due to memory I/O are counted in CPU utilization ? Here is more on - CPU Utilization is wrong.
I am a Mechanical Engineer by degree and Computer Engineer by profession. Here are the stories of self taught CS Engineers. - No CS Degree
The online library that collects education CS material from Stanford courses and distributes them for free. I particularly liked the Unix section. - Stanford CS Education library.
Book recommendation (courtesy: Alok). “High Performance Web Sites” - This book lists 14 specific rules to improve you client side performance.

Non-Technical:

What differentiates Professionals from Amateurs. - link
I love Emails. Unlike “instant” messages, they don’t pressure you to respond quickly without thinking much. Here is a great write-up on - Composing better mails.
A cool way of exploring realistic virtual Universe, travel from star to star, from galaxy to galaxy, landing on any planet, moon, or asteroid with the ability to explore its alien landscape. All on you computer. Check – Space Engine
A quote from a book - Mindwise

“More time together did not make the couples any more accurate; it just gave them the illusion that they were more accurate.”

Weekly Bullet #14 - Summary for the week

Mon, 22 Jul 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

“Advice for new developers, or Things I wish I had known when I started programming.” - Part1, Part2, Part3
As an IT professional you would have come across “10x Engineer” write-ups atleast once. Here is a myth buster about it. - “10x engineers”: Stereotypes and research
Brendan Gregg’s new upcoming book on Perfomance Tools - “BPF Performance Tools: Linux System and Application Observability (book)”
[Long article - Read selectively] - What every computer science major should know.
“Every linux networking tool I know” - Julia Evans

Weekly Bullet #13 - Summary for the week

Mon, 01 Jul 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

Talk by Brendan Gregg - Cloud Performance Root Cause Analysis at Netflix. YOW conference. - YouTube-Link. Length: 1hour
University of Helsinki are offering free course in AI. After finishing you’ll receive certificate you can add to your any profile. - link
Raspberry Pi 4 is available now! Also, here are the cool projects that can be built using Raspberry Pi. - link.
A list of pioneers in computer science - Wikipedia-Page

Non-Technical:

How to Be Great? Just Be Good, Repeatably - Article-Link
Documentary : Richard Feynman, more than a well known physicist, he is an amazing personality. This guy has had a great impact on my life. Here is a documentary on this amazing persons life - YouTube-link. - Length: 1hour.
Johy Ive, the Chief Design Officer of Apple is leaving Apple to form independent design company with Apple as client. - Link
A site that lists the “Top Sites” globally and country wise. I found Computer and Business category sites interesting. - TopSites.
An extract from the book that I am reading :

“Life (and our job) is difficult enough. Let’s not make it harder by getting emotional about insignificant matters”

[Performance Debugging] : Root causing "Too many open files" issue

Fri, 28 Jun 2019 00:00:00 +0000

Operating system : Linux

This is a very straight forward write-up on how to root cause “Too many open files” error seen during high load Performance Testing.

This article talks about:

The ulimit parameter “open files”,
Soft and Hard ulimits
What happens when the process overflows the upper limit
How to root cause the source of file reference leak.

Scenario :

During a load test, as the load increased, I was seeing failures in transaction with error “Too many open files”.

Weekly Bullet #12 - Summary for the week

Mon, 24 Jun 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

Performance Engineering book recommendation – “Systems Performance_ Enterprise and the Cloud by Brendan Gregg”

Three scientists published a paper proving that Mercury, not Venus, is the closest planet to Earth using Python. Checkout the amazing visualizations built using Python for the same. Article - link . Video - link.
What happens behind the scenes when you do a search in Google. - “How web works.”
Here are a bunch of great qualities that every senior-engineer (pre-manager / manager), ideally should poses. Check out – " What are the signs that you have a great manager? "
One of my mentors always says this - “You are paid for your thinking and problem solving abilities”. Here is a great compilation of websites which will help you hone these skills. – “A list of all problem solving websites.”

Non-Technical:

Here is another great Reading list. Note: Books are mainly Non-fictional / Programming related. – “Popular reading lists.”
This year marks the 50th anniversary of first ever Moon landing. Here is a super-cool website to relive Apollo 11 mission! – “Apollo 11 in Real Time”. Click on Join-in Progress button.
Wikipedia compilation of common misconceptions across Art & Culture, History and Science. Fun read. - “List of common misconceptions”
An extract from the book that I am reading :

“A person’s success in life can usually be measured by the number of uncomfortable conversations he or she is willing to have.”

Weekly Bullet #11 - Summary for the week

Mon, 10 Jun 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

Java comes with built-in Performance monitoring tools, which you might want to be familiar with - link
AWS costs every programmers should know - link . Also related info on all EC2 instances. - link
Various JVM options available, along with their descriptions. - “Java hotspot VM options”. There are over 100 options along with descriptions like below.

Weekly Bullet #10 - Summary for the week

Mon, 03 Jun 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

What will be Roles of a Performance Engineer in another year or two. Great write up. - “Trades of a Performance Engineer in 2020!”
MIT lectures on Advanced Data Structures. Also, the professor Erik Demaine is an amazing guy - “Advanced Data Structures (Fall'17)”. Side note: Worth checking comments on the original post - here
The process of preparing for Google interview, being rejected, re-prioritizing things and getting selected in Amazon. Author of this article went through this journey. - “Google Lost a chance to hire me… Finally, Amazon hired me !!!”
Understanding and changing the default for history command in Bash. - “Better bash history.”

Non-Technical:

Books recommended by over 100 founders and makers in tech. “Rework” is my all time favorite from the list. - " Founder Books"
I have written about Spaced repetition and Anki tool for the same. It does wonders and here is a write up on - “Tips for using Anki and Spaced Repetition in 2019”
A map of the US where city names are replaced by most Wikipedia’ed resident. Try zooming in and out. - “A People Map of the US”
An extract from the book that I am reading:

“Khaled Hosseini wrote The Kite Runner in the early mornings before working as a full-time doctor. Paul Levesque (page 128) often works out at midnight. If it’s truly important, schedule it. As Paul might ask you, “Is that a dream or a goal?” If it isn’t on the calendar, it isn’t real.”

Performance Bottleneck : High CPU Utilization vs High CPU Saturation

Wed, 29 May 2019 00:00:00 +0000

This article is more about a performance scenario that I found myself in, a few days ago, and my thought process about the same. It is about a situation when a Performance Engineer has to weigh the impact of CPU Saturation and not just CPU Utilization.

Scenario:

I was testing the Horizontal Scaling efficiency of an AWS EC2 instance, and at some points I was seeing low CPU utilization but high CPU Saturation (higher load averages).
Should I be spinning up new AWS instance because the CPU is saturated, although I have low CPU utilization (CPU % usage)?

Weekly Bullet #9 - Summary for the week

Tue, 28 May 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

Performance Check - List of commands to check Utilization, Saturation and Errors(USE method) at different components in a system. - “USE Method: Linux Performance Checklist”
Performance wise - “Why is it faster to process a sorted array than an unsorted array?”
Python has a long list of cool external libraries. But on the contrary, here are a list of - “Python built-ins worth learning.”
I am a big Check-list person! Recently I came across DevCheckList that lets create, share or collaborate on a checklist.
Here is an ultra black terminal I have been using for a couple of weeks now! - “Hyper”

Non-Technical:

If you love playing Chess, here are a bunch of - “Very Aggressive Openings.”
More often than not, full-time Remote work options look great. But just like everything, they have pros and cons. Here are learning from - “A Decade of Remote Work”
[Geeky - Highly Recommended] This blew my mind! Animation of - “507 mechanical movements” ! [ I did study Mechanical Engineering for 4years! :) ]
An extract from a book I am reading :

“In the beginning of your career, you spend time to earn money. Once you hit your stride in any capacity, you should spend money to earn time, as the latter is nonrenewable.”

Weekly Bullet #8 - Summary for the week

Sat, 18 May 2019 00:00:00 +0000

Hi All!

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

Performance Engineering and Tuning is equally important on Client side as it is on server side. Here is a free course on client performance optimization from a Google performance engineer. - “Website Performance Optimization”
A list of useful resource for debugging and optimizing Client Side Performance. -“Awesome Web Performance Metrics”
If you have recently moved to Python 3.X, here are a few things which you didn’t have in Python 2.x. - “Things you’re probably not using in Python 3 – but should.”
Here is a great dump of information on - “System Design.”
The shift in the hiring process in Software Industry is true! Here is a thoughtful article on the same. - “Senior Developers are Getting Rejected for Jobs”
Highly Recommended for a healthy laugh - “Let’s deploy to production!”

Non-Technical :

I have recently been exploring Board Games. Here is a great compilation for - “The Best 2-Player Board Games.”
I have been Tracking my expenses for 3 years now, and I review them by every month end. I use AndroMoney for tracking. It has both WebClient and AndroidApp. There are 100 other apps out there, but the point is, once you know where your money is going, you can control it.
I came across Kevin Kelly in one of the podcasts. He is movie buff and has a list of great movie recommendations. - “TrueFilm”.
An extract from the book that I am reading:

“It’s very interesting to observe who the top competitors pick out when they’re five rounds into the sparring sessions and they’re completely gassed. The ones who are on the steepest growth curve look for the hardest guy there—the one who might beat them up—while others look for someone they can take a break on.”

Weekly Bullet #7 - Summary for the week

Sun, 05 May 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

“Intermediate Vim “- has daily usable and highly productive Vim commands listed.
You have heard about git a lot, but don’t know where to get started ? - “Learn how to use Git (from basics)” . This would take your 30minutes.
“Introduction to Computation and Programming using Python.” - Course has been taken by a million people. You will learn to think computationally and write programs to tackle useful problems. The course is free to try. Also, here is “Intro video” about the course.
Here is how Python is used at Netflix. - “Python at Netflix.”
Want to do some web-scrapping with Python and need some ideas about the same. - Check the comments!

Non-Technical :

Have you ever had an idea of getting your personal website, but then didn’t spend enough time to make it happen ? Here is another reason to get back to that idea. - “You should have a personal website.”
A study on how much money professionals make across different fields. Study link. Results link. Note: Make sure to use filters for the results.
Practical career advice - “Career advice I wish I’d been given when I was young.”
A view on current state of Internet:

Shell Scripting for Performance Engineers and others - [Part 1]

Tue, 30 Apr 2019 00:00:00 +0000

Performance Engineers go through a set of manual tasks time and again. Be it for creating data for the load test, triggering of the test in a particular sequence / at a particular time or post processing of data collected after the test.
The general rule of thumb is - anything that takes more than 10 minutes and has to be done more than two times a week has to be automated. That is a minimum of 1040 minutes saved per year - 2 working days / year per person.
To achieve automation, although the world has come to Python & Scala for sophisticated solutions, quick and dirty Shell Scripts will never go out of style. I would not go for sophisticated / complex solutions, if the same can be attained in less than 20-lines of a quick Shell Script.

Weekly Bullet #6 - Summary for the week

Sat, 27 Apr 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

Time spent on Garbage collection in Java plays a very important role in fine tuning the Performance of an application. Here is a great talk on - “Understanding Java Garbage Collection and What You Can Do about It.”
Cheat sheet for Sorting Algorithms - It also explains each algorithm in details on how they work and what is their Big O cost. - Cheat Sheet Link.

Weekly Bullet #5 - Summary for the week

Sun, 21 Apr 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

Performance tuning Client side is equally as important as Server side. Here is a great article on – “How to Identify expensive functions using the Chrome DevTools CPU Profiler.”
Majority of us never get beyond surface-level understanding of how git works. Here are a few – “Tips for a disciplined git workflow.”

Weekly Bullet #4 - Summary for the week

Sat, 13 Apr 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

In Linux, perf command can give you a lot of power for instrumenting the system if you know how to use it. Here is a “not so short” description on how to use it. Note: A post about perf commands is in the pipeline & I will post it shortly ! – “Perf command in linux.”
Bash is more powerful than Black Magic (if that exists!). Here is a 3 part series on some powerful “Bash one-liners.”
Developer surveys usually give an insight on what most people are working on / the next Big Thing! Here are the results of “Stackoverflow Developer Survey Results 2019.”
Top documents and papers on Hacker News in 2019. Updated weekly. – “Hacker News Papers.”
If you have a student email, take advantage of the Github student pack and others. “Github educational free pack.”

Non-Technical :

I greatly enjoyed the podcast. – “How Seth Godin Manages His Life – Rules.”
Note : It is a 123minute long podcast. I consumed it at 2x play speed during an overnight journey. One of the take aways for me – “Most of the people keep playing with the cards they have got instead of moving to a different table with different cards. Don’t get stuck. Move to a different table.”
Have you tried Stoic philosophy ? If not, I highly recommend it for general life wisdom. If a Stoic philosophical book like “Letters from a Stoic” is too much for you, then there is an app – “The Stoic” which will fill you with one Stoic quote a day!
So there was the first image of Blackhole this week. This explains the efforts involved in making Blackhole image a reality. – Youtube link.
An amazing extract from a book that hit me:

“When you are very competitive, you get good at the thing you are competing with people on. But it comes at the expense of losing out on many other things. It is important to understand and accept the consequences.”

Weekly Bullet #3 - Summary for the week

Fri, 05 Apr 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical:

If you had only 60 secs to check the performance of a linux system, here would be a check list to find the bottlenecks ! “Linux Performance Analysis in 60,000 Milliseconds”
We all know the importance of cutting and parsing only required parts of an output in programming. Awk on a whole can do much more than that. Here is - “Why you should learn just a little Awk”
I found this resource on github which is suitable for almost every programmer, System and Network administrators, DevOps, Pentesters and Security Researchers. - “The Book of Secret Knowledge”
On a lighter note - Remember the first time you opened a Vim editor and didn’t know how to exit ? Here is a funny milestone on same ! “Stack Overflow ~ Helping One Million Developers Exit Vim”

Non-Technical :

Learning never ends. So it is good to know the best ways of learning. Here is a discussion on the same. - “Ask HN: What are your best learning methods/hacks/tips? "
Learning is important true, but we forget what we learn if there is no repetition. I use Anki for spaced repetition. Anki works wonders. Reviewing notes multiple times based on the ones you mark Easy, Medium or Difficult. - “Anki”
Quote I’m pondering :

“I don’t give explanations anymore, and I’ll catch myself when I start giving explanations like ‘Oh, I’m sorry, I can’t make it. I have a doctor’s appointment. I’m really sick. I broke my leg over the weekend’ or something. I just say, ‘I can’t do it. I hope everything is well.’ "

Weekly Bullet #2 - Summary for the week

Thu, 28 Mar 2019 00:00:00 +0000

Hi All !

Here is the weekly summary of Technical / Non-Technical topics that I found very resourceful.

Technical :

Programming notes on almost every language. I have learnt half of the Python I know from here. “Programming Notes for Professionals books”
The reason I love Linux is, there are tools available for peeking in to the performance of every component. Below cheat sheet lists the set of commands to look in to different components.

About / Contact

Mon, 25 Mar 2019 00:00:00 +0000

Thanks for joining me!

I’m a Software Engineer with a passion for making software applications and products faster, more reliable, and cost-effective.

Most of my current interests revolve around the Observability of systems and System Architecture for building efficient and resilient systems. I also spend time exploring cost efficiency on AWS Cloud.

My first love - Linux
Old friend - Python / Neovim / CLIs
New neighbour - Golang / CNCF / LLMs

The Journey Begins

Mon, 25 Mar 2019 00:00:00 +0000

Thanks for joining me!

Good company in a journey makes the way seem shorter. — Izaak Walton

Weekly Bullet #1 - Summary for the week

Mon, 25 Mar 2019 00:00:00 +0000

Hi All !

This is an idea that I have been planning to try for quite sometime now. A summary of what happened over the week. A weekly bullet would come out every Saturday and it would cover:

What interesting stuff happened in Tech or Non-Tech world over the week –(Not NEWS)
Extracts from the books that I am reading (somethings which have hit me hard)
Resources Tech/Non-Tech that I might have come across.

So here is the First of something New.