I've recently been looking into locally hosting some LLMs for various purposes, I haven't specced out hardware yet. Any good resources you can recommend?
Kind of, it's a standalone system with the hardware integrated - kinda like Google Coral with a Raspberry Pi.
I’ve recently been looking into locally hosting some LLMs for various purposes, I haven’t specced out hardware yet. Any good resources you can recommend?
Not really, sorry - I haven't gone too deep into LLMs beyond simple use cases. I've only really used llama.cpp myself.
Ah, gotcha - the Jetson devices are definitely robowife capable lol. I suspect it is mostly used for things like AI enabled electronic devices, but it is possible to use ROS for robotics!
you are very welcome! i’ll see what i can do about creating more, but this one had me laughing out loud once i figured out why the jetson wouldn’t boot anymore heh
I can package it up and send it to you for informational purposes, but please don’t attempt to recreate the bug, as we have all seen the Reanimator, haven’t we?
Funhole
Active