Overview
Folio Lab is the only agentic-aware quantitative SaaS offering in the Indian market that does portfolio optimization on Indian equities and mutual funds, using state-of-the-art optimization techniques, from standard practices to machine learning. The following is a deep dive into the architecture of this piece of software, which I have been developing for the past 2+ years. Originally it started out as a personal litmus test for me to implement the learnings from the Portfolio Optimization class I had taken as a graduate student, but later it evolved into the full-blown computational platform that you see today.
This blog post explains the architecture of the platform I have built, and traces a request from a client (an AI agent, an API client, or a browser) through the secure network to the API, all the way to the end result, which contains the weights, metrics, charts, etc. (stored in R2 and the PostgreSQL database). All of this happens asynchronously, with dedicated workers and queues, which allows us to scale each of the components independently, on both the horizontal and the vertical axes.
Why I left the public cloud
Ever since 2018, when I was first introduced to the cloud native world and building applications in a public cloud environment, I had written all my applications, both professionally and personally, in the Big 3 cloud environments of AWS, GCP and Azure. For the v1 iteration of this project as well, I had made use of the AWS free tier and hosted the backend on a t2.micro instance, with the API being stateless and no storage options offered. But with the recent increase in memory pricing, the general cost of public compute increasing YoY, and the price floor rising for even the smallest compute tiers, I had decided to switch to a hybrid cloud model, with R2 being my object store, Neon being the Postgres database, and compute (API, workers and queues) fully hosted on a DigitalOcean Droplet.
But soon even this architecture started showing its limitations. The first was Neon to go away, when they changed their free compute tier and I quickly ran out of the compute hours on their platform. Then the Droplet fees also started adding up, since I had switched to a machine with higher memory (2 vCPU, 8 GB RAM, about $48 per month give or take). At this pace I was going to quickly run into bottlenecks on my platform and wouldn't be able to scale and keep up with the traffic. So instead I went to a different approach of owning all my compute myself (CAPEX instead of the OPEX I was following till now) and came up with a new hybrid architecture, that not only brought my compute and data closer but also gave more capacity and control over the scaling (horizontal and vertical) based on the new features being added to the platform and the traffic.
With the emphasis on owning the compute and the data, and only storage relegated to R2, I had started searching for old Dell, HP, Lenovo and Asus mini PCs in the secondary market, so that I could buy one, maybe two, and connect them over a network so that I have a decent amount of compute, memory and storage. But the problem with that approach is that the hardware which was available in the Indian secondary market was quite outdated, on both the Intel and the AMD side of things, and the relatively new ones were expensive. So I decided to bite the bullet and purchase a new machine that would fit my criteria. I purchased a Lenovo M70s Gen 6 small form factor PC with an Intel Core Ultra 7 265 vPro processor (20 cores, 20 threads) and 16 GB of memory, and installed Ubuntu Server 24.04.5 LTS on it.
The system, end to end
This architecture can be split into three parts, moving left to right in Fig 1:
- The client-side requests, which originate in the public internet (The clients).
- The Cloudflare layer, which is the only part of the system that the internet can see. It holds the tunnel, which carries requests to and from the API server, and the R2 object store, which holds each run's artifacts (The ingress tunnel, Artifacts).
- The rightmost part, the core of the architecture, which holds the API, the workers, the queues, the database, the monitoring and alerting tools, and other ancillary services, all of which run on the server and expose no ports to the public-facing internet (The API, The Database, The Queue and the Workers, Monitoring).
The life of a run
The easiest way to understand how these parts fit together is to follow a single optimization run from the moment it is submitted to the moment its results show up on screen.
- Submission. A client sends
POST /jobswith the tickers, the date range and the optimization methods. The client can be the web app in a browser, a script with an API key, or an AI agent over MCP. All three end up in the same service layer, so there is exactly one code path that accepts a run, whichever door it came through. - Ingress. The request lands on Cloudflare and travels down the tunnel to the server. The tunnel is opened from inside the server, so there is no inbound port on the machine for anyone to find or attack.
- Admission. The API authenticates the caller, checks that their plan has quota left for this kind of run, and writes the run into Postgres with the status
queued. Only after that row is committed does the API put the run's id on the queue. It then replies straight away with therun_id. - Inputs. A worker picks the
run_idoff the queue, reads the full configuration back from Postgres, and marks the run asrunning. Every input in that configuration comes straight from the user: the assets, the date range, the methods, the risk-free rate and any constraints on the weights. For the risk-free rate, the user picks the 91-day, 182-day or 364-day T-bill or the 10-year government bond, and the worker averages that yield over the same date range. - Optimization. The worker loads the price history of those assets from the database and turns it into daily returns. Everything else the math needs is computed from those returns, and each method computes only the measures it uses. The mean-variance methods need the expected returns and the variance-covariance matrix, the tail-risk methods look at the losses and drawdowns in the return history, and the hierarchical methods group the assets by how closely they move together. With those in hand, the worker solves each requested optimization.
- Persistence. The weights, the metrics and a compact copy of the chart data go into Postgres, so the first view of a result never has to wait on object storage. The heavier outputs, such as the full result data and the Excel workbook, go to R2 while the run shows the status
uploading. Once both are written, the run becomessucceeded. - Delivery. All this while, the client has been polling
GET /jobs/{run_id}. When the status flips tosucceeded, the results page reads the numbers from Postgres and fetches each file from R2 through a short-lived presigned URL. - Follow-up work. Anything the user asks for later on the same run gets its own queue: a PDF report goes to
pdf_reports, and a Monte Carlo simulation goes tomonte_carlo, where it is dequeued and picked up by a separate worker.
The rules behind it
That trace is really a handful of design decisions, and they are worth stating on their own, because the rest of this post keeps coming back to them.
- The API admits, the workers compute. An optimization can take anywhere from a second to several minutes. Holding an HTTP request open for that long would tie up the API and break behind any proxy with a timeout, so the API only ever does the fast part.
- The database is the source of truth, not the queue. The message on the queue carries nothing but the
run_id; everything else lives in Postgres. Because the row is committed before the message is sent, a worker can never receive a run that does not exist. If Redis refuses the message, the API marks the run as failed on the spot, instead of leaving it stuck asqueuedforever. - Each kind of work gets its own queue and its own worker. A Monte Carlo simulation can run for minutes and use a lot of memory. On a shared queue it would sit in front of short optimizations and make every user wait. Separate queues mean each worker can be sized and scaled for the work it actually does.
- Rows in Postgres, files in R2. Small, queryable data goes to the database. Large blobs go to object storage, which lives off the server. If the machine dies, every artifact that was ever produced is still there.
- One box, one file. Everything on the server, all thirteen containers (Postgres, Redis,
cloudflared, the API, the two workers, the scheduler and the six monitoring services), is described in a single Docker Compose file.
The clients: the Browser and MCP
Folio Lab has two front doors, and both lead to the same place. The first is the website, which is what most people use. The second is MCP, the Model Context Protocol, which lets an LLM such as ChatGPT or Claude use Folio Lab on the user's behalf. There is also a quieter third door, the API itself, for anyone who would rather script their runs with an API key.
The Browser
The website is a Next.js application hosted on Vercel. It holds no data of its own, and every page is drawn from calls to the API. A run always starts with a handful of choices, and all of them belong to the user. The first is the set of assets: Indian stocks or Indian mutual funds, found by name or by ticker. A single run cannot mix stocks and funds. US stocks are available as well, but in a run of their own, because a single run cannot mix the two markets. The user then picks the history window, which is the start and end dates of the price history the optimizers learn from, and any number of the 38 optimization methods. All the chosen methods run side by side on the same data, so that their outputs are comparable.
The user also chooses the benchmark the portfolio is measured against. The choice runs from the Nifty 50 and the Sensex down to the midcap and smallcap indices, and the user can measure against either the plain price index or the total return index, which adds the dividends back in. On top of that, a run can optionally carry a rolling backtest of the portfolio and a Monte Carlo simulation of it.
The newest part of the form is the constraints, which let a user write the kind of limits a real portfolio mandate contains. The simplest constraints put a floor and a ceiling on a single stock, or on every stock at once, or leave some stocks out altogether. Group constraints go further. A user can say "no more than 30% in banking", using a sector list that they save once and reuse across runs. A group limit can also be measured against part of the portfolio instead of the whole, such as "at most 35% of the equity part", or it can apply to each member of a group separately instead of to the group's total.
Every constraint restricts the weights that the respective method can choose. Either the optimizer finds a numerical solution, or the problem is infeasible, and both outcomes are shown to the end user. To avoid wasting a run on constraints that make the optimization infeasible, the front end is smart enough to check them against the chosen assets while the user edits the inputs. It shows the range each group can actually reach, and which assets will be dropped for lack of price history. A run can also report what its constraints cost, by comparing each constrained portfolio with the one the same method builds with no constraints at all. Constraints are available on the Enterprise plan.
MCP
The API also hosts an MCP server, and any LLM that is compatible with MCP can connect to it. Today that means ChatGPT and Claude. The user adds Folio Lab as a connector, signs in with their own Folio Lab account and approves the access on a consent screen, much like signing in to an app with Google. From then on the LLM acts as that user, with that user's plan and limits, and the user can revoke its access at any time from their account page.
The LLM needs the same inputs as the website, but nobody has to fill in a form to give them. A user can simply say "optimize HDFC Bank, Infosys, TCS and ITC over the last five years, and keep banking under 30%". The LLM looks up the tickers from the names and makes the tool call for the optimization, constraints included.
Where MCP really earns its place is after the run finishes, because the LLM does not just hand back a table of weights: it interprets them. When it explains a metric, it works from Folio Lab's own documentation and links to the page the explanation came from, instead of relying on what it happens to remember. When it compares runs, it warns if one run had constraints and the other did not, because a Sharpe ratio earned under constraints and one earned without them are not a fair comparison.
The LLM also draws the charts itself. The server never sends images; it returns the chart data, and the LLM builds the chart inside the conversation. When it reports a Monte Carlo result, it states the probabilities as conditional on the model, the inputs, the settings, the engine version and the random seed of that run, and never as a forecast. Every answer about a run links back to the full results page on the website, where every chart and the downloadable reports live.
The website has an LLM of its own, in a chat panel on the site, and it uses exactly the same MCP server and the same tool calls for optimization, backtesting, Monte Carlo simulation, reporting and everything else. So there is only one set of tools to maintain, and an answer in ChatGPT, in Claude or on the website comes from the same code.
The ingress tunnel
Every request from the internet, whether it comes from a browser, an LLM or a script, is addressed to api.foliolab.ai. That name points at Cloudflare, not at my server. Cloudflare handles the HTTPS connection at its own edge and then sends the request down a tunnel to the server.
The tunnel works the other way round from a normal server. Usually a server opens a port and waits for the internet to connect to it. Here, a small program called cloudflared runs in a container next to the API and connects out to Cloudflare, and requests travel back down that connection. So the server has no open ports, and nothing on its network has to forward traffic to it. A port scanner pointed at it finds nothing to talk to.
Because every request arrives through the tunnel, the API would otherwise see cloudflared as the sender of every request. The real client address travels in a header that Cloudflare adds, and the API only believes that header when the request comes from the tunnel. A caller cannot fake their address to slip past the rate limits.
Only the API's traffic uses the tunnel. The website's pages come from Vercel, and the result files come from R2 through Vercel, so the tunnel carries only the requests that actually need the server.
The API
The API is a FastAPI application, and it is the only thing on the server the tunnel talks to. It has three jobs: it works out who the caller is and what they are allowed to do, it keeps the records, and it hands the heavy work to the workers. There is one kind of identity for each door. The website signs in with an email and password or with Google, and gets a short-lived access token plus a refresh cookie that the page's own JavaScript cannot read. A script uses an API key, and an LLM uses an OAuth 2.1 token issued through the consent screen described above. All three resolve to the same user, so the plan, the limits and the run history are the same whichever door a request came through. Each plan is a row of limits and feature flags in the database, such as how many runs a month, how many Monte Carlo runs, and whether constraints are allowed, and the API checks these before it accepts any work. The flags that switch a feature on fail closed, so a mistake in the plan table can only take a feature away, never hand one out by accident. Payments come in from Razorpay through a webhook, and the API moves the user onto the plan they paid for. Everything that is not a computation is ordinary CRUD over Postgres: the run history with its names and tags, saved portfolios, sector lists, saved comparisons, API keys, connected LLMs and the account itself, and deleting a run also deletes its files from R2. The API returns the numbers of a run from Postgres, but it does not carry the files. When a page wants a file, the API checks that the run belongs to the caller and then signs a link to the file in R2 that expires after a short time, which makes the API the gatekeeper for the files, not the pipe they flow through. The MCP server is mounted inside the same application, so its tools call the same service code as the website's routes, and nothing is written twice. The website's chat panel is served from here as well: the API passes the conversation to the language model, runs the tools the model asks for, and streams the answer back to the browser as it is written. Because a request never waits on an optimization, a single API process is enough, and the API protects itself by refusing oversized request bodies and by rate-limiting the sign-in routes.
The Database
The database is PostgreSQL 17, and it runs in a container on the same machine as everything else. It did not start out there. For most of the project's life it lived on Neon, a managed Postgres service, in their Singapore region. When production moved to the M70s in Hyderabad, nothing in the code changed, but every query from the API now had to travel to Singapore and back, and that round trip measured 50 ms. The API makes several small queries in sequence to answer a single request, so those 50 ms add up quickly. The results page, for example, spent about 2.7 seconds of server time across the API calls it makes, and almost all of that time was spent waiting on the database.
To find out what a local database would actually buy, I restored a copy of the Neon database into a tuned Postgres on the M70s, checked that every table matched on its exact row count, and then ran two containers of the production API image side by side, identical except for the database each one pointed at. The first row below times one query on its own. The other rows were measured over HTTP, with real authentication, as the average of five requests after a warm-up:
| Request | Neon, Singapore | Postgres on the M70s |
|---|---|---|
A bare SELECT 1 | 49 ms | under 0.1 ms |
| A run's full result | 888 ms | 32 ms |
| A run's configuration | 262 ms | 3.7 ms |
| The billing summary every page loads | 895 ms | 7.4 ms |
| The whole results page | 2,694 ms | 58 ms |
| The whole rolling backtest page | 1,814 ms | 50 ms |
Most of the time on Neon was spent on the network, not in the database. Two requests show this most clearly. GET /runs/{run_id}/monte_carlo lists the Monte Carlo simulations that belong to an optimization run, and GET /runs/{run_id}/backtests lists its standalone backtests. For a run that did not have any Monte Carlo simulations or backtests yet, both requests returned an empty list, and each response was only 60 bytes. Even so, each of those requests took about 260 ms on Neon, which is roughly five round trips of 50 ms each, with almost no work done in the database. On the M70s, the same two requests took 6.3 ms and 3.6 ms. Across every call it makes, the results page became 46x faster, and the rolling backtest page became 36x faster. The API now reaches Postgres over the local Docker network. Most of the delay that is left on a page is the round trip between the user and Cloudflare, and that part was the same before the move.
Running Postgres myself also means tuning it myself. The M70s has 16Gb of memory shared between twelve other containers, and the database is about 4.2 GB, so the memory settings are sized for that. shared_buffers is 2 GB, which keeps the working set in Postgres's own cache, and today about 97% of block reads are served from it. effective_cache_size is 6 GB, which tells the query planner how much of the database the operating system's page cache is likely to hold as well, so that it prefers index scans where they pay off. work_mem is 32 MB per sort or hash operation, enough for the analytical queries to sort in memory without letting a burst of connections exhaust the machine. maintenance_work_mem is 1 GB, so that an index rebuild or a restore never spills its sort to disk. That last setting has a trap in it: by default, autovacuum inherits the same limit, and three autovacuum workers at 1 GB each could take 3 GB of the machine between them. autovacuum_work_mem is therefore pinned separately at 256 MB, which is plenty for routine vacuuming.
The disk is NVMe, and the planner is told so. random_page_cost is lowered to 1.1 from its default of 4, which assumes a spinning disk where a random read is far more expensive than a sequential one. On an SSD it barely is, and leaving the default makes Postgres avoid index scans that would actually be faster. effective_io_concurrency is raised to 200 so that Postgres issues many reads at once, which an SSD handles well. On the write side, wal_buffers is 16 MB and max_wal_size is 4 GB, so checkpoints happen less often, and checkpoint_completion_target of 0.9 spreads each checkpoint's writes across most of the interval, instead of flushing everything in one burst that stalls the queries running at the same time.
None of this is bought with durability. synchronous_commit stays on, so a transaction is on disk before the API is told it committed, and the database was created with data checksums, so a corrupted page is detected instead of being silently read. track_io_timing is on, so the time a query spends waiting on the disk shows up in the statistics and can be told apart from time spent computing. The server's time zone is pinned to UTC, because the monthly usage limits are computed partly in SQL and partly in Python, and the two have to agree on where a month ends. Finally, the container has no memory limit, on purpose: a hard limit on a database turns a spike in memory into the kernel killing Postgres, which is an outage, and the machine has the headroom to absorb one instead.
The Queue and the Workers
The queue is Redis, and the workers use Dramatiq, a Python task library, to take work off it. Because a message carries only a run ID, a Redis restart that lost its messages would lose no run data; the run would only need to be put back on a queue. Redis requires a password even inside the compose network.
There are three queues, split by the kind of work rather than by who asked for it. jobs_default carries optimizations and standalone backtests, pdf_reports carries the PDF reports, and monte_carlo carries the simulations. The split matters because the three kinds of work behave very differently. An optimization takes anywhere from a second to a few minutes, a PDF report is short, and the largest Monte Carlo simulation the platform accepts runs for over nine minutes and uses more than a gigabyte of memory.
The worker container dequeues from jobs_default and pdf_reports, and it runs one process with four threads. Threads might look like a poor fit for numerical Python, but the heavy lifting happens inside NumPy and its linear algebra libraries, which use several cores on their own. In practice, a single worker process spans many CPU cores at once. In my testing, memory has been a bigger constraint for these workers than compute, because they need to hold several key pieces of data in memory at once to apply the constraints and find a solution numerically. Anecdotally, across various runs, I have observed the peak sustained memory use linger around 1.13 GiB, and in anticipation of higher usage in the future, I have capped this worker's memory limit at 4 GiB.
The mc-worker container dequeues from monte_carlo only, and it runs two processes with one thread each. The simulation engine is CPU-bound NumPy that mostly uses a single core. With one process, the container sat at about 100% CPU for whole runs on a machine with 20 threads, which was the one clear case of unused capacity in the stack. Two processes run two simulations at once. The largest simulation the platform accepts (12 assets, 100,000 paths, 40 years, with walk-forward re-optimization) peaks at 1,181.5 MiB, so two of them need 2.31 GiB, and the 4 GiB limit keeps that at 58%.
Failures are handled differently on each queue. An optimization is not retried automatically: if it fails or runs past its time limit, the run is marked as failed with the reason, so the user sees the failure instead of waiting on a run that will never finish. A PDF report gets one retry. Monte Carlo and backtest runs also have time limits and no automatic retry, and a recovery task re-enqueues any of them that went stale, such as a Monte Carlo run whose lease timed out, or one that stalled in the queue without being picked up. The scheduler exists as an independent service for this. It dequeues nothing, and on a fixed interval it re-enqueues all backtest and Monte Carlo runs that went stale. Each run it recovers is claimed with a conditional update in the database, so two copies of the scheduler running at the same time can never pick up the same run twice.
Artifacts
Everything a run produces that is not a row in the database is a file in Cloudflare R2, an S3-compatible object storage service. For an optimization run, that means the full result data, compressed as gzipped JSON, and an Excel workbook with the weights and the metrics; a run with a rolling backtest adds a second workbook for the backtest. Monte Carlo simulations and standalone backtests write their own result files and workbooks in the same way, and every file an attempt writes carries that attempt's lease token in its name, so a retried attempt can never overwrite the files of an earlier one. The PDF reports are the one exception to the rule that a worker writes its files as part of the run. A PDF is slower to render, so it is only made when someone asks for it, on the pdf_reports queue, and it lands in R2 next to the rest of the run's files.
The files are served only through signed links. When a page needs a file, the API checks that the run belongs to the caller and returns a link to the file in R2 that expires after one hour. The browser then fetches the file through a small proxy route on the website on Vercel, because a browser cannot read R2 directly from another origin, so the file never passes through the API or the tunnel. R2 also charges nothing for egress, which matters for files that are downloaded again every time someone opens a result. When a user deletes a run, its files in R2 are deleted with it.
Monitoring
Prometheus watches everything on the server. Every 15 seconds it scrapes metrics from every part of the system, and it keeps them for 30 days. The metrics come from four kinds of source. node-exporter reports the machine itself: CPU, memory, disk and network. cAdvisor reports the same things for each container, so a worker that is creeping towards its memory limit is visible well before it reaches it. redis-exporter reports on Redis, including how many messages are waiting on each queue. And the API and both workers publish their own application metrics: request rates, latencies and errors from the API, and how many runs each worker processed, how long they took and how many failed.
Prometheus also evaluates a set of alert rules against those metrics: the API's error rate and latency, a disk filling up, a queue whose backlog is not draining, a worker that is down or close to its memory limit, and more. When a rule fires, Alertmanager groups related alerts together and sends them to Discord through a Discord webhook. A critical alert goes out within ten seconds and repeats every hour until it is resolved, while warnings are grouped for longer and repeat every four hours. The webhook URL works like a password for posting to the channel, so it is kept as a Docker secret: a file mounted into the Alertmanager container alone, instead of an environment variable that every container could read. The request to Discord is the only thing Alertmanager sends out of the machine.
Grafana turns the same metrics into dashboards. There is one for the machine, one for the health of the services, one for the queues and the workers, one for runs, one for Monte Carlo capacity and one for billing. The machine dashboard shows the host's CPU, memory, disk and network in detail, down to the use of each core, and the service health dashboard puts each container's resource use next to the metrics of the service inside it. So when something is slow, it is quick to see whether the cause is the host, a single container or the application itself. The dashboards are defined as files in the repository and loaded when Grafana starts, so they are reviewed and versioned like the rest of the code.
Conclusion
Folio Lab runs on one machine that I own: a Lenovo ThinkCentre M70s that runs the API, the workers, the queues, the database and the monitoring as thirteen containers, with Cloudflare in front of it and R2 holding the files.
On DigitalOcean, the $32 per month droplet with 4 GB of memory was not enough, so I had to move up to the $48 per month droplet with 8 GB, both before 18% GST, and then to this machine, which replaced that bill with a one-time purchase. Buying the machine outright gets me more compute than a year of the $48 per month droplet, which comes to $576, or about $680 with GST. Moving the database onto the same machine made the pages that read it 36x to 46x faster.
The price of that is that everything depends on one machine. There is no failover, so a hardware fault, a power cut or an outage of its internet connection takes the platform down until it is fixed. For a platform at this stage, I think that is the right tradeoff.