Aqueduct Turns 1 🥳¶
Today exactly 1 year ago - on March 31, 2025 - we pushed the first commit to Aqueduct. Aptly named "Initial commit". Since then a lot has happened and we wanted to reflect on the progress, the current state and the future plans (the backlog only grows longer and longer).
Origin Story¶
The "Data Science Team" was founded in October 2024, which now feels like the stone age. The goal was very broad, we had to find out where to go from there and there was no clear path to follow.
First, we planned to use classical machine learning (ML) where possible. Large Language Models (LLMs) were still the new thing back then and we figured we will still heavily rely on classical ML approaches for the first use cases we identified. However, LLMs turned out to be very useful as they truly are a "jack of all trades". You basically hit the ground running as they can adapt to most use cases very easily.
In the true dataLAB spirit we were of course going to host the models ourselves. But what about all the infrastructure around them? Specifically, the AI gateway as this is the single point of entry and controls who has access and to what. Even in a bare-bones setup as we had, this was a necessary component as this was to be the "face" of LLM-as-a-Service.
When looking for off-the-shelf solutions, we only found US startups with absurd valuations. Their open source offerings were their dumbed down version of the enterprise plans with critical features missing. We had a sales meeting and they tried to upsell us for several tens of thousands of Euros a year just to have SSO.
Naturally, we quickly decided we are going to build it ourselves. And so the race was on.
Development¶
Of course we were going to make the project open source on GitHub. It felt like everyone was doing different things and there was a lot of uncertainty at the time. As we already said, it was quite unclear what the best path forward was. Open sourcing was a way to position us: This is what we ARE doing!
We wanted the tech stack to be as simple as possible. We did not want to reinvent the wheel and we wanted something that comes with "batteries included". Django it is! And turned out to be the perfect choice, even though we had not used it before.
At first we started coding everything by hand (I was learning Django after all), but being part of the group that most people now colloquially call the "AI guys", lends itself to experiment with AI coding tools.
When first developing the UI I saw glimpses of what coding agents could be (and are doing now to a limited extent). I believe I was using the then new GPT 4o in Cursor, which had quite a generous free tier then. Everything I did was "one-shotted" (mind you these were simple HTML pages). But HTML/CSS is basically NP-hard for me. I can tell immediately if something looks reasonable but I can't for the life of me figure out how a flex grid works. So this literally made me 10x faster during that stage.
From there, we started adopting agentic coding more and more. We now moved to OpenCode, a CLI coding agent, which was much easier to use. We learned a bit what works when and where, where it helps and where it fails. But that is a story for another time.
Development will continue either way.
Looking Ahead¶
The mission is still to provide a platform for AI at TU Wien. Be it for AI-enabled services or coding agents, for researchers or staff, for experts or people just starting out.
This is hard. Which is to say digital sovereignty is hard. And expensive both in time and money. This year and beyond we will see how far we can push this. Do we want to be locked in to contracts with multi-trillion dollar corporations? Or will we have an AI platform where we have full control over everything and the know-how and agency to shape how we use it?
So, what is the plan? This year the focus is on scaling. We recently hit a milestone where we first served more than 1 billion tokens in a week through Aqueduct and the underlying infrastructure, and we just got new hardware to increase that further.
As adoption for agentic coding and AI in general increases so do our compute needs.
This comes with its own set of challenges of course, which we are currently addressing on both the gateway and infrastructure side. We have an upcoming cooperation planned with the Austrian Scientific Cluster (ASC) to better scale in the future. Furthermore, we are in the process of bringing Aqueduct to multiple Austrian universities as part of the aiCampus project. So, Aqueduct is hopefully growing in the future.
The only thing left to say is thanks for all your support and your patience when things did not always work as planned.