Does anyone have any experience running a translation server in a “serverless” environment?
Be it Functions as a Service (Lamba or Azure Functions) or a PaaS serverless container service (Elastic Container Service or Azure Container Apps)?
I think it would be a neat way to deploy something like this if your situation is that there’s a lot of idle time, or even just very irregular loads. It could presumably handle infinite scalability while also scaling to zero/a baseline. Cold starts could perhaps be mitigated by pre-sending a quick request upon some sort of initial trigger action (user activity) to warm the server.
FaaS don’t seem to have granular cpu and ram controls, but they do have request concurrency controls for scaling function instances. I suspect there’s other limitations as well for FaaS - can you install/use a C++ tool like CLanguage2? Youd probably even need to refactor an entire translation application to be able to work on it well, let alone cheaply.
But with a PaaS, like Azure Container Apps in particular, you can just install a docker container and be on your way. You can have an always-warm container, if desired. And you can set the CPUs and RAM available per container.
They even offer pretty generous free tiers - 1 million or more requests etc…so you might not even end up paying anything at all!
I found this article series talking about why and how they moved from Azure Functions to ACA. Bye bye Azure Functions, Hello Azure Container Apps: Introduction
I’ll likely explore this option at some point, but I’d love to hear any thoughts anyone has on any of this!