Karan Sharma

When you are debugging a production incident, you are forming hypotheses, testing them against evidence, ruling things out, trying not to confuse correlation with causation, and making triage calls under time pressure. The hard part is the reasoning – figuring out why the system…

I wanted a browser terminal at terminal.mrkaran.dev that works from laptop, tablet, and phone without special client setup. The stack that works cleanly for this is ttyd + tmux. Architecture# Browser -> Caddy -> ttyd -> nsenter -> su - karan -> tmux(main) Two decisions matter mo…

I recently set up LiteLLM with AWS Bedrock as the LLM provider. The docs cover the happy path, but there are a few gotchas that cost me some debugging time. This post covers what I learned, from basic setup to per-project cost tracking with Application Inference Profiles. Model …

In my previous post, I introduced Gullak, an expense tracker I built to categorize transactions using LLMs. While the initial prototype used SQLite, I’ve since made a fundamental shift in how Gullak stores data. It now uses the ledger-cli format—a plain-text accounting standard …

The industry is currently obsessed with defining standards for how Large Language Models (LLMs) should interact with software. We see a proliferation of SDKs, function calling schemas, and protocols like MCP (Model Context Protocol). They all aim to solve the same problem: bridg…

About eight months ago I wrote about Logchef – a log viewer I’d been building to scratch my own itch with log exploration at work. Back then it was basically a nicer way to query ClickHouse without writing raw SQL every time. Today I’m shipping v1.0, and it’s evolved into someth…

About a month ago, I downloaded my CIBIL report expecting a routine check. Instead, I found loans from lenders I had never interacted with, written-off accounts, overdues from fintechs I had never installed, and even two-wheeler loan enquiries. I don’t even ride a bike. My credi…

Everyone is worried that AI will replace programmers. They’re missing the real revolution: AI is turning everyone into one. I’ve been noticing a new pattern: people with deep domain knowledge but no coding experience are now building their own tools. Armed with AI assistants, th…

Introduction# For the past five years, I have maintained a homelab in various configurations. This journey has served as a practical exploration of different technologies, from Raspberry Pi clusters running K3s to a hybrid cloud setup and eventually a cloud-based Nomad setup. Ea…

I recently spent some time debugging a WireGuard tunnel that was acting weird. The handshake was successful, pings worked perfectly, but any TCP connection failed with connect: no route to host. Classic misleading error message. The routing was fine. The Setup# Server with a pub…

So, for the last 3-4 months, I’ve been busy building Logchef. This tool basically grew straight out of my day job managing logs at Zerodha, where I’ve been managing logs for almost half a decade. I wanted to share a bit about how Logchef came to be. Like many, we journeyed throu…

I’ve been introduced to Nix by my colleagues at work. Being a Linux user for over a decade and a serial distro hopper, I was curious to learn more about it. I’d seen Nix mentioned before, but the comments about its steep learning curve made me wonder if the effort was worth it. …

I’ve been playing badminton more regularly since the start of 2025 - almost 4-5 days a week. I recently moved to a new part of the city, which meant I couldn’t play with my old friends anymore. PlayO has been super helpful for finding games with new people. On PlayO, a host crea…

My Obsidian vault has gotten quite messy over time. I’ve been dumping notes without proper frontmatter, tags were all over the place, and some notes didn’t even have proper titles! I needed a way to clean this up without spending hours manually organizing everything. I’d been pl…

2024 was indeed an important year for me as it marked several significant milestones. Quite happy with how this year was! Here’s my reflection on this memorable year. Life# Got married to the prettiest and dearest Saumya 💗 Did my first international trip, exploring Europe Bough…

Just yesterday, GitHub announced integrating Claude 3.5 Sonnet with Copilot. Interesting times ahead. In my experience, Claude has been remarkably better than the GPT-4 family of models for programming tasks. I’ve tried a bunch of tools like Cursor, Continue.dev but finally sett…

I recently discovered Outline a collaborative knowledge base. I wanted to self-host it on my server, but the mandatory auth provider requirement was off-putting. My server is on a private encrypted network (Tailscale) that only my approved devices in the tailnet can access, so I…

A couple of weeks ago, I decided to start logging and tracking my expenses. The goal was not to record every minor purchase but to gain a general insight into where my money was going. In this post, I’ll dive deep into the behind-the-scenes of building Gullak—an expense tracker …

Last month, I did a wonderful trip travelling through the scenic landscapes of Switzerland. My wife and I were in Lucerne and had scheduled a day trip to Mt. Titlis for the next day but were wondering what to do that evening. After strolling along the Chapel Bridge and enjoying …

I have an upcoming trip to Europe, which I am quite excited about. I wanted to set up a Tailscale exit node to ensure that critical apps I depend on, such as banking portals continue working from outside the country. Tailscale provides a feature called “Exit nodes”. These nodes …

Earlier this week, I had stumbled upon 1brc, which presents a fun task: loading a huge text file (1 billion lines) in Java as quickly as possible. The One Billion Row Challenge (1BRC) is a fun exploration of how far modern Java can be pushed for aggregating one billion rows from…

Introduction to SadServers# Recently, I stumbled upon sadservers, a platform described as “Like LeetCode for Linux”. The premise is: you are given access to a full remote Linux server with a pre-configured problem. Your mission is to diagnose and fix the issues in a fixed time w…

This blog post is ignited by the following Twitter exchange: I don’t take the accusation of unsubstantiated argument, especially on a technical topic lightly. I firmly believe in substantiated arguments and hence, here I am, elaborating on my stance. If found mistaken, I am open…

At $dayjob, we use AWS Pinpoint to send out SMS to our customers. We’ve also written a detailed blog post on how we use Clickhouse + vector stack for our logging needs. We additionally wanted to store the delivery logs generated by the Pinpoint service. But like with anything el…

To set the stage, it’s crucial to understand what we mean by “bridge networking”. In a nutshell, it is a type of network connection in Linux that allows virtual interfaces, like the ones used by virtual machines and containers, to share a physical network interface. With Nomad, …

You know those budget freaks? People who log and categorise every Rupee they’ve spent over the month? The financially sane people? I am definitely not one and I suck at it. I moved cities a couple of months back and had some big ticket spends off late, mostly financed by credit …

At work, we use a Vector pipeline for processing and shipping logs to Clickhouse. We also self-host our SMTP servers and recently started using Haraka SMTP. While Haraka is excellent in raw performance and throughput, it needed an external logging plugin for audit and compliance…

A few months ago, a proposal for adding a structured logging library in Go was introduced by Jonathan Amsterdam. At present, Go has a minimal and bare-bones log package which works all right for basic use cases. However, the current library has a few shortcomings that this propo…

CoreDNS is an extensible DNS server (which is actually a fork of Caddy v1) that can be used to serve DNS records for a domain. It is written in Go and is very easy to extend. It has a plugin system that allows you to write your own plugins to extend its functionality. In this po…

I’d been mulling around reading a computer science paper and implementing a project based on it. Distributed systems, Networking and Databases are some of the things that fascinate me a lot. However, I had been looking to implement a more approachable project to avoid getting in…

Application orchestrators like Nomad, Kubernetes etc., allow you to deploy multiple applications on the same host. Logs are stored on the underlying node wherever the applications are run. However, it’s pretty common to treat these instances as ephemeral if they’re a part of an …

which is a non-standard/non-POSIX compliant program. I faced many issues getting which to work in a chroot environment (Nomad). So basically, which is a simple shell script program to find out the dependency by searching the $PATH (which is what makes it less deterministic). It’…

On June 28, I received a text from Anand nudging me to submit a CFP for IndiaFOSS. Until then, I’d not planned to attend IndiaFOSS because travelling to participate in a conference didn’t seem enticing. However, the thought of visiting Bangalore (a city where I spent five years …

At work, my colleagues and I stumbled upon a hair-pulling networking issue involving a specific problem when connecting to a Kafka cluster. We use the franz-go library in our Golang applications to interact with an external Kafka cluster. These Go apps are hosted inside a Nomad …

Nomad is a general-purpose cluster orchestrator and scheduler. Up until Nomad 1.3 was released, it had no native support for discovering other applications running in the cluster. This is sort of a very elementary requirement when scheduling tasks in a cluster. Nomad relies on C…

I’ve often been asked how to expose public and private services running on DigitalOcean droplets/RPis when self hosting apps. Most people don’t have access to a static IP for their RPis. I felt I’d summarize my approach to this in this blog post and hope it’ll be useful for othe…

ClickHouse is an extremely performant columnar DB used for fast analytical processing. ClickHouse supports data replication using Apache Zookeeper which needs to be deployed externally. While Zookeeper works well after you’ve tuned it properly, it was still an additional mainten…

Recently, I’d posted on Twitter that my feed has become messy overtime. Maybe I followed some accounts that were of no interest, maybe Twitter’s algos don’t really know what to show to me. Whatever, I wanted a fresh start. And, I’ve done this in past. I’ve removed all the accoun…

Today I faced an issue that questioned my sanity. Since I didn’t find many “related” issues on StackOverflow/Google-fu except a lone GitHub thread where a kind stranger hinted at what could be the issue, I am writing about it here in the hopes (although I don’t wish this torture…

This week I was occupied with optimising a Golang program I’d written at work. I wanted a way to reproduce the issue under heavy load on my development environment and Load Tests are a good way to do that. The service in question is a RESTful API so it’s relatively easy to use a…

I’ve been toying around Clickhouse and Vector at my day job and find both of these tools pretty interesting at what they do. A short summary for those unaware of these tools: Vector helps you build a pipeline for collecting, transforming and processing different kinds of observa…

Over the past 2-3 years, note-taking apps have become all the rage. Note-taking is an extremely subjective topic and a lot of it depends on the individual’s workflow. There’s no one size fits all and maybe that justifies the ever-expanding landscape of such apps. I’ve tried a fe…

It’s been a long time since I’ve written a post on Hydra (my home server). I use Hydra as a testbed to learn new tools, workflows and it just gives me joy to self-host applications while learning something in return. History# A brief history of how Hydra’s setup evolved over tim…

There are some things you expect to just work. Sadly trying to make Gitlab Runner with AWS ECR turned out to be quite a daunting task and the little documentation in this area doesn’t help. There’s even a 4 years old issue and everyone there is echoing the sentiment that this is…

In this post we’ll see how to create a simple job queue in Golang. There are tonnes of libraries and posts out there doing overly complicated stuff, however if your need is pretty minimal or you want to understand the concepts from ground up, this post aims to do just that and n…

Have you ever been stuck in a situation where you’ve spent countless hours digging through lot of options, overthinking their pros/cons and having a tough time to decide? If yes, I’ve been in the same boat myself and the more I think about this, I think it’s causing some serious…

I’ve been writing on this blog for about 2 years now. This has been the longest I’ve stuck on to the same technology stack for my blog. I’ve previously jumped from a Jekyll based static site to a Medium blog before finally settling for Hugo. I’ve been using Hugo since 2018 but I…

Terraform has a straightforward way of importing existing records (managed outside Terraform) via terraform import command. The usage is documented here and works well if you have a handful of records to import. However when you work with custom Terraform modules and have a whol…

Twitter is an amazing thing! For all the shitposting and meme-ing that’s done there, there are some really cool people you get to interact with which wouldn’t have been possible IRL. I happened to stumble upon Swapneel last year in Bangalore at a meetup. He occasionally posts ab…

I like monitoring stuff. That’s what I do at work and when my home ISP started giving me random problems and I decided it would be nice to monitor my home network as well. There are a couple of ways to go around this, a very popular and OSS solution is SmokePing. SmokePing is wr…

For those of you who are new to my blog, I had written about my home server setup (hydra) a few months back. Since then I’ve tweaked the setup a bit and made some changes to how I organise/deploy applications inside my cluster. I’ll be talking about the updates made so far and t…

One of the primary advantages of deploying workloads in Kubernetes is seamless application discovery. Intra cluster communication becomes easy with the concept of Service which represents a Virtual IP backing a set of Pod IPs. For example, if vanilla service needs to talk to a c…

Well, the title is self explanatory so let’s begin! First off, we need to ensure that the StorageClass which was used to provision the PVC has the correct configuration. From the official docs: You can only expand a PVC if its storage class’s allowVolumeExpansion field is set to…

The Motivation# Sometime back I had to access a Kubernetes API server which was firewalled to a private VPC network. I didn’t want to setup a separate bastion instance just to access this cluster, cause TBH bastions are kinda redundant in K8s as every task can be performed throu…

Quite often you’d need to check if a port on a target node is opened or blocked by firewall. I’ve always used telnet to test that but it has a few drawbacks: Need to use dirty hacks in shell scripts to auto close the connection. Also, telnet outputs some errors to /dev/stdout in…

For the longest time I’ve had these commands in my .gitlab-ci.yml file for a K8s CD pipeline: ... - kubectl apply -k overlays/prod - echo "Waiting for 15 seconds for pods to be restarted" && sleep 15 - kubectl get po ... So, basically I apply the changes to cluster using kubectl…

kubekutr was born out of my frustration of organising K8s resource manifests files/directories. For the uninitiated, K8s lets you hold the state of your cluster declaratively as “manifest” files. K8s does so by a lot of asynchronous control loops to check whether the desired sta…

Dig is a DNS lookup utility developed by BIND which helps a lot while troubleshooting DNS issues (which are more common than you probably think #hugops). I use dig fairly often and thought to write an introductory guide on how you can use dig with some practical examples that’ll…

In this post, I’d like to share my experience and learnings about configuring a deployment pipeline for Kubernetes. I’ll be using Gitlab CI/CD and AWS EKS to demonstrate the concept, but the core idea remains the same: all changes must come declaratively from a single source of …

EKS uses a custom authenticator tool called “aws-iam-authenticator”. The basic idea is to make the auth flow in EKS easier by using the tools you already use in AWS. To wrap your head around the flow, consider three separate entities: A kubernetes resource (entire namespace, spe…

So, I got hold of 2 Raspberry Pi4 (still limited stocks in India) recently and wanted to build a Kubernetes cluster. Don’t ask why cause that would be pointless. I’ve little experience with a managed Kubernetes workload (Amazon EKS, which btw deserves its post 😝) but never real…

If I have nothing to hide then I have nothing to show either When you visit a site from your browser, there are several entities which have access to this information in one request-response cycle. Starting from the router you connect to, to the DNS resolver you use to resolve t…

So, off late, I’ve been dabbling a lot with Docker to explore the world of containerization (too late to the party, eh?). I plan to write about some common docker gotchas. The plan is to document these learnings so they might help someone getting started with docker and also ser…

Managing Kong using UI# Konga is an unofficial project which is basically an UI for Admin API of Kong. This post is about how to setup Konga and configuring an upstream service. We will also add an authentication layer to our upstream. Docker to the rescue# We will use Docker to…

Kong# Kong is an API Gateway, which basically reverse proxies every incoming request to the upstream URL. It is pretty useful if you have a lot of internal services which need to talk to each other (through HTTP) but you don’t want to keep managing the authentication layer, rate…

Prometheus configurations can turn into a mess in no time if you have a lot of different jobs scraping different targets. Certainly you can use tools like jsonnet to keep your YAML files DRY but this post is not about that. I initially started off by writing one job to scrape a …

Makefile is an awesome tool to group together a bunch of different rules and automate your build process. Makefile is used by make which is essentially a file generator tool. It is generally used to compile and build programs from source by following rules listed in the Makefile…

I am writing this post exactly after 30 days of the last one, caught up with some work and I couldn’t write a new post, apologies! To make up for the delay, I have some absolutely gold reccos this time which I believe you will like it. Isaac Gracie - reverie# Tash Sultana - Jung…

Since the past one week, I’m having troubles in connecting to Spotify from India and my #1 source for discovering music is gone. I’ve started using Saavn but it has crappy recommendations and too much of “Bollywoody”/crap shit thrown all over. Also, it requires location access t…

Ive looped through Nescafe Basement like 100 times the past two weeks and still cant get enough of it. This is the kind of music I like, the kind that just grows on you until you loop endlessly and then hate it Anyway, the following is a list of songs that I have discovered and …

Supervisor Events# I had a seemingly simple task which was to receive notifications any time a process managed by Supervisor restarts. I wanted a generic solution where I could get notifications for any change in the process state. Supervisor Events saved my day, although I woul…

I listen to a lot of music artists while working or otherwise. I plan to share some of the gems I discover each week. Thanks to Spotify and Youtube suggestions, I keep discovering a lot of indie artists and underrated titles. I hope you give them a listen and spread some good mu…