Skip to content

Ramalama Serve Model Swap #2080

@bmahabirbu

Description

@bmahabirbu

Feature request description

lets say we have a model store with models A,B,C,D

Ramalama serve no argument will create a serve container that mounts all the models with the proxy able do the model swap

Ramalama serve A,B will create a serve container that only mounts A and B still able to model swap

Ramalama serve A will create a serve container with no proxy not able to swap (for security reasons)

Suggest potential solution

@engelmi has did some amazing work creating the ton of the groundwork for this so it just needs to be implemented now

Have you considered any alternatives?

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions