Overview
Folio Lab is the only agentic-aware quantitative SaaS offering in the Indian market that does portfolio optimization on Indian equities and mutual funds, using state-of-the-art optimization techniques, from standard practices to machine learning. The following is a deep dive into the architecture of this piece of software, which I have been developing for the past 2+ years. Originally it started out as a personal litmus test for me to implement the learnings from the Portfolio Optimization class I had taken as a graduate student, but later it evolved into the full-blown computational platform that you see today.
This blog post explains the architecture of the platform I have built, and traces a request from a client (an AI agent, an API client, or a browser) through the secure network to the API, all the way to the end result, which contains the weights, metrics, charts, etc. (stored in R2 and the PostgreSQL database). All of this happens asynchronously, with dedicated workers and queues, which allows us to scale each of the components independently, on both the horizontal and the vertical axes.
Why I left the public cloud
Ever since 2018, when I was first introduced to the cloud native world and building applications in a public cloud environment, I had written all my applications, both professionally and personally, in the Big 3 cloud environments of AWS, GCP and Azure. For the v1 iteration of this project as well, I had made use of the AWS free tier and hosted the backend on a t2.micro instance, with the API being stateless and no storage options offered. But with the recent increase in memory pricing, the general cost of public compute increasing YoY, and the price floor rising for even the smallest compute tiers, I had decided to switch to a hybrid cloud model, with R2 being my object store, Neon being the Postgres database, and compute (API, workers and queues) fully hosted on a DigitalOcean Droplet.
But soon even this architecture started showing its limitations. The first was Neon to go away, when they changed their free compute tier and I quickly ran out of the compute hours on their platform. Then the Droplet fees also started adding up, since I had switched to a machine with higher memory (2 vCPU, 8 GB RAM, about $48 per month give or take). At this pace I was going to quickly run into bottlenecks on my platform and wouldn't be able to scale and keep up with the traffic. So instead I went to a different approach of owning all my compute myself (CAPEX instead of the OPEX I was following till now) and came up with a new hybrid architecture, that not only brought my compute and data closer but also gave more capacity and control over the scaling (horizontal and vertical) based on the new features being added to the platform and the traffic.
With the emphasis on owning the compute and the data, and only storage relegated to R2, I had started searching for old Dell, HP, Lenovo and Asus mini PCs in the secondary market, so that I could buy one, maybe two, and connect them over a network so that I have a decent amount of compute, memory and storage. But the problem with that approach is that the hardware which was available in the Indian secondary market was quite outdated, on both the Intel and the AMD side of things, and the relatively new ones were expensive. So I decided to bite the bullet and purchase a new machine that would fit my criteria. I purchased a Lenovo M70s Gen 6 small form factor PC with an Intel Core Ultra 7 265 vPro processor (20 cores, 20 threads) and 16 GB of memory, and installed Ubuntu Server 24.04.5 LTS on it.
The system, end to end
This architecture can be split into three parts, moving left to right in Fig 1:
- The client-side requests, which originate in the public internet (The clients).
- The Cloudflare tunnel, which forwards requests to and from the API server, and the R2 object store, which stores each request's artifacts (The ingress tunnel, Artifacts).
- The rightmost part, the core of the architecture, which holds the workers, the queues, the database, the monitoring and alerting tools, and other ancillary services, all of which run on the server and expose no ports to the public-facing internet (The API, The queue and the workers, Monitoring).