Pidoku
Extending vLLM

Extending vLLM

ExpertTopic3 lessons~1h 50m

Lessons, in order

About this topic

The last topic turns from reading vLLM to changing it. It covers the registered extension points that make a fork unnecessary, what a model has to look like to run inside the engine, and how to stay current with a project that ships every two weeks. Each lesson doubles as a review: explaining what a plugin may replace, or what a model must not do, restates what the engine itself is responsible for.

#LessonThe question it answers
01Plugins and Extension PointsWhat can I replace without forking, and which interfaces are stable?
02Adding a ModelWhat must a model class look like, and what does the engine take over?
03The Rust Frontend, and Keeping UpWhere is vLLM heading, and how do I verify anything I have learned?

↑↓ navigate↵ openesc close

drag to pan · scroll to zoom