Networking for AI inference model serving on all backends