Serverless Translation Server?

Does anyone have any experience running a translation server in a “serverless” environment?

Be it Functions as a Service (Lamba or Azure Functions) or a PaaS serverless container service (Elastic Container Service or Azure Container Apps)?

I think it would be a neat way to deploy something like this if your situation is that there’s a lot of idle time, or even just very irregular loads. It could presumably handle infinite scalability while also scaling to zero/a baseline. Cold starts could perhaps be mitigated by pre-sending a quick request upon some sort of initial trigger action (user activity) to warm the server.

FaaS don’t seem to have granular cpu and ram controls, but they do have request concurrency controls for scaling function instances. I suspect there’s other limitations as well for FaaS - can you install/use a C++ tool like CLanguage2? Youd probably even need to refactor an entire translation application to be able to work on it well, let alone cheaply.

But with a PaaS, like Azure Container Apps in particular, you can just install a docker container and be on your way. You can have an always-warm container, if desired. And you can set the CPUs and RAM available per container.

They even offer pretty generous free tiers - 1 million or more requests etc…so you might not even end up paying anything at all!

I found this article series talking about why and how they moved from Azure Functions to ACA. Bye bye Azure Functions, Hello Azure Container Apps: Introduction

I’ll likely explore this option at some point, but I’d love to hear any thoughts anyone has on any of this!

have u tried docker? I think u can set up docker env.

Yes, docker is used for azure container apps and elastic container service. Not for FaaS.

I’m just curious if anyone has experience trying to run a translation server in a serverless environment as opposed to a standard server environment.

not to my knowledge because of the latency to instantiate things.

Yeah, fair enough. I’ll probably test it but likely will just have to use a normal server that’s always up. Azure container apps could still work though - leave an idle instance and spin up new ones based on some reasonable trigger.