Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : A few interesting and hidden features of Python - link Below are the BCP tools that can be used for digging in to Performance analysis of memory parameters on a Linux machine. More in BPF Performance Tools book by Brendan Gregg. Book : BPF Performance Tools How does Database indexing work? - here How to get the most out of your 1:1s - here At around 8-10years of experience, career branches in to either Engineering management or Technical Staff Engineer. Here are a few stories of Staff Engineers and their journey. [Self-endorsement]: A tiny 50lines of code tool for getting my highlighted quotes from my Fav books - here Non-Technical : [Highly recommended]: First-principles thinking is a competitive advantage because almost no one does it. More here The importance of annual health checkup - Seema’s True story of battling cancer - here I always thought about a social platform but based on Books. Booqsi is trying to do the exact same. It is still in early beta though. More details here A website that removes things from images in seconds- Magic Eraser What Got You Here Won’t Get You There: Book Summary - here Podcast : Hugh Jackman - on Daily routine, intuition, meditation, decision making and more. link A quote that the struck right chords: “If you are not willing to take responsibility for your situation, you cannot use your situation as an excuse either.” ...

There is no better pleasure than the Joy of solving your own problems. This write up is not to show off coding skill(there is hardly any code in this tool), but to show the ease with which anyone can build tools to solve problems these days. Problem statement: How to retain the most out of the books we read? Maybe receiving daily reminders with extracts from the books? I consume books mainly in the digital format(via kindle/calibre). I have a lot of highlights in these books which I want to be periodically reminded about. I felt, if I take out 6hours in reading a book, and completely forget all learnings in next 6months, then that’s not efficient. ...

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : Memory leaks on client side - the forgotten side of web performance. Link A list of helpful patterns/commands on "_sed_" command - link A “Streaming availability” api to lookup which show/movie is available in which OTT in 60+countries. Something to explore for a fun weekend project - link You remember the times, when there are 10 terminal sessions opened on your system and you wanted to know “at what time did I run that command” ? Ya, article addresses the same problem - link PS: If you use zsh, there a few themes which come in-built with this feature. Wordle puzzles are crazy popular in the last week or so. Here is a python project to solve the puzzles. link Non-Technical : Completing part-time master’s in CS while on a full time job! This article was so inspiring also reminds, how much time we waste in general. - link Drop a raindrop anywhere in the world and watch where it ends up. A fun site - link Rocket engines, they deal with a lot of heat right? How come they don’t melt? Detailed geeky explanation - here An interesting Thread on Human psychology fact. link [Image below]: “Stop focusing on the black lines behind you. Start focusing on all of the green lines before you.” ...

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : CPU utilization is wrong - PS : idle waits are counted in the %CPU. Python : Comprehensive Python Cheatsheet Finished coding, but waiting for PR to be review/approved ? - The Pull Request Paradox Best practices can slow your application down - Best Practices vs Required Practices - link Profiling and analyzing performance of Python programs - link Also, since it is Monday: ...

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : P99 CONF (centered around low-latency, high-performance design) recordings are available here Python f-strings can do more than you thought. Video here Web3 is not a hype. An article on what is Web3 and decentralized internet here. Also, a podcast by Tim Ferris with Chris Dixon and Naval Ravikant – The Wonder of Web3. Another one of those – “20 Things I’ve learned in my 20 years as a Software Engineer.” Most helpful sed one liners - here How WhatsApp scaled to 1 billion users with only 50 Engineers. - here The “Advent of code” will be live on December 1st. Here is some history about it. Discussion thread. Non-Technical : Traveling(rather walking) without money! Although old(1998) event, still shows world is not that bad of a place. Link “MY PENNILESS JOURNEY” There was this mega-thread on twitter for “one book that changed the way you see the world”. - The consolidated list from the thread on BooksChatter here. Matthew McConaughey addressing University of Houston outgoing students – “5 Rules for the life” (Rule #1 is my fav) Extract from a book: “If you think about the biographies you read or the documentaries you watch about the greats in various fields, this same pattern of Addictive, Passionate behavior surfaces. Jazz saxophone great John Coltrane reportedly practiced so much that his lips would bleed.” ...

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : eBPF Summit is live now - Recording of the Keynote and live summit here Conference talk - USENIX LISA2021 Computing Performance: On the Horizon by Brendan Gregg - here [video - 41mins] A project for visualizing codebase - here Self healing systems - the real end goal of observability - here Python3 - Reverse Engineering Tips - here Non-Technical : Great article on - “How to think: The skill you’ve never been taught” - here An addictive trading game in your browser - Paper trade - here People no longer trust each other. Why? And how can we fix it? An interactive guide to the game theory of trust - here What tiny purchases have disproportionately improved your life? - Thread link Extract from a book: “But we had two choices,” I said. “Throw our hands up in frustration and do nothing, or figure out how to most effectively operate within the constraints required of us. We chose the latter. ...

As a Performance Engineer, time and again you will come across a situation where you want to profile CPU of a system. The reasons might be many; like, CPU usage being high, you want to trace a method to see its CPU cost or you suspect CPU times for a slow transaction. You might use one of the various profilers out there to do this. (I use yourkit and Jprofiler). All these profilers report the CPU costs in terms of CPU Time, when you profile the CPU. This time is not the equivalent of your watch time. ...

Here are a bunch of Technical / Non-Technical topics that I came across recently and found them very resourceful. Technical : All recording from PyCon US 2021 are up on Youtube here. My fav is Keynote by Robert Erdmann about rebuilding 5 µm resolution picture of Rembrandt’s painting “The Night Watch” from 18th century with Python. Rembrandt’s painting “The Night Watch” from 18th century “Docker For The Absolute Beginner” course. This is offered free on kodekloud.com . The same course was taken by over 97,000 students on Udemy. datefinder is an amazing python module for location date out of different date formats in a string. Here is a short video about the same. Book recommendation - “BPF Performance Tools” - By Brendan Gregg. BPF-based performance tools give you unprecedented visibility into systems and applications, so you can optimize performance, troubleshoot code, strengthen security, and reduce costs. ...

These are the self-notes from managing 100+ node ES cluster, reading through various resources and a lot of production incidents due to unhealthy ES. Memory Always choose ES_HEAP_SIZE 50% of the total available memory. Sorting and aggregations both can be memory hungry, so enough heap space to accommodate these is required. This property is set inside the /etc/init.d/elasticsearch file. A machine with 64 GB of RAM is ideal; however, 32 GB and 16 GB machines are also common. Less than 8 GB tends to be counterproductive (you end up needing smaller machines), and greater than 64 GB has problems in pointer compression. CPU Choose a modern processor with multiple cores. If you need to choose between faster CPUs or more cores, choose more cores. The extra concurrency that multiple cores offer will far outweigh a slightly faster clock speed. The number of threads is dependent on the number of cores. The more cores you have, the more threads you get for indexing, searching, merging, bulk, or other operations. Disks If you can afford SSDs, they are far superior to any spinning media. SSD-backed nodes see boosts in both querying and indexing performance. Avoid network-attached storage (NAS) to store data. Network The faster the network you have, the more performance you will get in a distributed system. Low latency helps to ensure that nodes communicate easily, while a high bandwidth helps in shard movement and recovery. Avoid clusters that span multiple data centers even if the data centers are collocated in close proximity. Definitely avoid clusters that span large geographic distances. General consideration It is better to prefer medium-to-large boxes. Avoid small machines because you don’t want to manage a cluster with a thousand nodes, and the overhead of simply running Elasticsearch is more apparent on such small boxes. Always use a Java version greater than JDK1.7 Update 55 from Oracle and avoid using Open JDK. A master node does not require much resources. In a cluster with 2 Terabytes of data having 100s of indexes, 2 GB of RAM, 1 Core CPU, and 10 GB of disk space is good enough for the master nodes. In the same scenario, the client nodes with 8 GB of RAM each and 2 Core CPUs is a very good configuration to handle millions of requests. The configuration of data nodes is completely dependent on the speed of indexing, the type of queries, and aggregations. However, they usually need very high configurations such as 64 GB of RAM and 8 Core CPUs. Some other important configuration changes ...

When you come across a system which is misbehaving, majority of the times the first metrics that we look at is CPU usage. But do we really understand what CPU usage of a system tells us ? In this article let us try and understand what X % usage of a system really means. One of the easy ways to check on CPU is “top” command. The “%Cpu(s)” metrics seen above is a combination of different components. ...