I’m working on an internal tool. Maybe 30-40 “customers” total. I say it’s production because it has to be reliable.
We just don’t want to rent a GPU for this little thing. It draws up reports once a day, so it’s okay if it takes a couple mins. It’s work that took a single person maybe 2 hours to do before.
I’ll need to look into triton, I haven’t heard of that yet!
If you have any resources for running models in production that you’d be willing to share, I’d appreciate them.
We just don’t want to rent a GPU for this little thing. It draws up reports once a day, so it’s okay if it takes a couple mins. It’s work that took a single person maybe 2 hours to do before.
I’ll need to look into triton, I haven’t heard of that yet!
If you have any resources for running models in production that you’d be willing to share, I’d appreciate them.