Hi All, @greg Invited me as I’ve recently launched “Mighty Inference Server” https://max.io …I’ve been part of the ANN search community for several years now, and built Mighty for teams using Pinecone and similar vector search engines.
I’d really like to hear what you all are currently using for inference, and what problems you have. Nothing solves every problem but we focus on specifically self-hosted and fast inference for teams that don’t want to spend months building expensive GPU deployments, and want inference to be 100% unlimited (compared to costly Inference-as-a-Service APIs like OpenAI and Cohere). Infinity also gave lots of folks sticker shock (estimated at $20k per year) - which is fine for enterprise but is out of reach for most small to medium size teams.
It’s nice to meet you all and thanks for the warm welcome Greg .