axiotic/ogma-small
Sentence Similarity • 8.6M • Updated • 111 • 4
Aniket Maurya
aniketmaurya
AI & ML interests
Computer vision, NLP, MLOps
Recent Activity
liked a model 2 months ago
axiotic/ogma-small liked a model 7 months ago
pyannote/segmentation-3.0 upvoted an article over 1 year ago
Tensor ParallelismOrganizations
liked a
model 2 months ago
liked a
model 7 months ago
upvoted an article over 1 year ago
Article
Tensor Parallelism
huseinzol05
• • 21 updated a
model over 1 year ago
liked a
model over 1 year ago
published a
model over 1 year ago
replied to singhsidhukuldeep's post over 1 year ago
woohoo thanks for checking LitServe @singhsidhukuldeep ! LitServe now has OpenAI API-compatible endpoint and you can also serve a LLM using vLLM engine with LitServe so you get both speed + flexibility.
reacted to singhsidhukuldeep's post with 🚀 over 1 year ago
Post
944
Just tried LitServe from the good folks at @LightningAI !
Between llama.cpp and vLLM, there is a small gap where a few large models are not deployable!
That's where LitServe comes in!
LitServe is a high-throughput serving engine for AI models built on FastAPI.
Yes, built on FastAPI. That's where the advantage and the issue lie.
It's extremely flexible and supports multi-modality and a variety of models out of the box.
But in my testing, it lags far behind in speed compared to vLLM.
Also, no OpenAI API-compatible endpoint is available as of now.
But as we move to multi-modal models and agents, this serves as a good starting point. However, it’s got to become faster...
GitHub: https://github.com/Lightning-AI/LitServe
Between llama.cpp and vLLM, there is a small gap where a few large models are not deployable!
That's where LitServe comes in!
LitServe is a high-throughput serving engine for AI models built on FastAPI.
Yes, built on FastAPI. That's where the advantage and the issue lie.
It's extremely flexible and supports multi-modality and a variety of models out of the box.
But in my testing, it lags far behind in speed compared to vLLM.
Also, no OpenAI API-compatible endpoint is available as of now.
But as we move to multi-modal models and agents, this serves as a good starting point. However, it’s got to become faster...
GitHub: https://github.com/Lightning-AI/LitServe
- 1 reply
liked a
model about 2 years ago
upvoted an article about 2 years ago
Article
PaliGemma – Google's Cutting-Edge Open Vision Language Model
- +1
merve, andsteing, pcuenq
• • 288 liked 3
models over 2 years ago
liked a
model almost 3 years ago
liked a
model about 3 years ago
liked a
dataset about 3 years ago
liked a
Space almost 4 years ago