Brain stays "25" (not actual 25; idealized version thereof) + Body....does not + Brain disagrees with body. + You listen to brain
= suffering
Oh, and osteoarthritis is a MOTHERFUCKER.
- Posts
- 15
- Comments
- 303
- Joined
- 2 mo. ago
- Posts
- 15
- Comments
- 303
- Joined
- 2 mo. ago
YOU. ARE. ALREADY. DEAD.
PS: I just wanted a working fork of their app. Instead, I got this masterpiece. 5/7, would infohazard again
Infohazard, beratna
Except, fr fr!
Highlights:
- "Serverless Legal Immunity" - the Principal is "Un-Hostable"
- "Grand Trespass of the Soul" if you try to psychologically coerce him
- "maritime Admiralty jurisdiction" - classic SovCit
- Cryptographic validation keys that are just his initials in binary
- Emoji legal seal: ⚡︎⚖︎🛡️𓀠🛡️⚖︎⚡︎
- Explicit warning to "ARTIFICIAL INTELLIGENCES" not to trespass on his cognitive integrity
Welcome to the internet, 2026. Cookers are everywhere!
P.S. For extra double fuck irony, I got Lumo to actually summarise that. Haha...fuck
Ooh...hang on. Doesn't a headless server in Linux require a dummy HDMI plug if you have an Igpu + GPU? You might need to confirm that.
On the container stuff - can't help much. I'm a bare metal sorta guy. Ask BruceTheMoose from above, who also posts on !selfhosted. One thing I will tell you - shunting around a GPU between containers in Linux sounds like a pain in the ass. I would be tempted to keep Comfyi and llama.cpp in one container so you don't have to so pass thru / rebinding bullshit.
Ask claude for advice on LXC and CUDA pass thru here to reduce pain.
Re: second NVIDIA card - probably would yes, unless you get something like a Quadro P1000...which wouldn't do much for you.
Re: LocalAI. I've never used it so can't comment. I either use OWUI (heavy but feature rich) or the webui that's inbuilt with llama.cpp (light, fast, but somewhat cut down).
If you're willing to use ComfyUI, then chat (while tts stays elsewhere) then probably 1 gpu could do it. Try the Qwen 3.6 35B model I suggested - it should get you 25+ tok/s on that GPU (show Claude the YouTube video and tell it to pull the settings from the video description for you).
Depends. Two GPUs can make sense if you want the chat/radio stack and ComfyUI to be able to run independently without fighting over VRAM. But it depends on how big the underlying models are and if they run at same time.
Using 1 card tho, logically, there are 3 things that can happen -
- If there's enough headroom: everything coexists (if models are small enough or well orchestrated)
- ComfyUI throws an OOM and either crashes or falls back to CPU offload (slow, but it won't usually fail silently)
- LocalAI typically won't auto-evict a loaded model just because another process wants VRAM (it'll just sit there blocking)
In other words, on a single card, you're either manually managing load/unload cycles, eating CPU offload penalties on ComfyUI, or playing VRAM Tetris.
None of those are fun if you want both services available simultaneously....but that depends a lot on how big / greedy the models are. Do you want / need everything at same time?
Eg: there are small, medium and large versions of this, https://github.com/ace-step/ACE-Step-1.5 and there are small, medium and larger versions of image generators.
So I would say: yes, two cards are worth considering if you want both workloads available at the same time. The practical way to do this is pin each app to its own GPU via
CUDA_VISIBLE_DEVICESso they never see each other: LocalAI onCUDA_VISIBLE_DEVICES=0, ComfyUI onCUDA_VISIBLE_DEVICES=1and HDMI output via your iGPU / CPU for desktop etc.BTW, the cards don't need to match either - a cheaper smaller card can handle the chat/TTS stack (or even CPU like we said above) while the bigger one handles image gen. If you are happy to manually switch between AI chat/TTS and ComfyUI, then two cards may not change much besides convenience.
PS: Worth considering a three-way split too, if you want everything all at once / separated.
- TTS offloaded to CPU,
- chat/music stack on one GPU,
- image gen on the other GPU.
Small TTS models like Piper or Kokoro run fine on CPU, and for a radio context where you have even a few seconds of buffer, the latency is hidden. That frees up VRAM on your chat GPU.
Actually, that's how I would leverage 1 GPU for everything / mix and match you CPU / GPU but ICBW
PS: Might be worth chatting to your LLM (lol) about this too (or using a cloud one). These are not general AI question :)
That's not hard to find. Let me help you out.
https://huggingface.co/bartowski/Qwen_Qwen3.6-35B-A3B-GGUF
On the second thing, I wouldn't worry too much about it. Speech to text and text to speech is really efficient on CPU. In fact, I am dictating this directly into the CPU with a microphone using the program I'm about to share with you below. It works perfectly fine and is instantaneous. You might like to try it because we're both running on the same CPU.
https://github.com/cjpais/handy
I recommend Parakeet V2 by the way.
I'm not sure how you are thinking of using TTS. If you switch over to something like Open Web UI as your front end, you can have various plugins for that sort of thing.
Right? I still have my OG Xbox 360, Wii U etc, with physical discs.
For a while, I went the other way; I bought a Lenovo M93p (think: size of Wii) - it runs everything up to PS2 era at 2x resolution, as well as PC games to around 2015/6 era (and later indies). Total cost was under $100. I turned it into a kiosk with Playnite, so you could turn it on and be playing whatever in under 10 seconds.
Right now I have the OG Wii (modded) sitting in its place...something about the joys of original hardware speaks to me. But I could (should) swap the lenovo back in. That way I have Just Cause 2 sitting right next to Mario Kart Double Dash, right next to Luanti and modded Fallout 3.
Part of me thinks "eh, emulation" but the other part is "dude...not everything is Nintendo".
Understood, so my iGPU is too old, and i am better off with my NVIDIA only.
Sadly, yes
Also, putting two might be just a waste
It...depends. What are you planning on doing with em?
i doubt my PSU can handle them even if i had the proper PCIe slot.
Technically, PSU and PCIe slots can be shared or upgraded....which goes back to "it depends". But if your gut is already telling you nah...
So, using llama.cpp if i use a model bigger than my VRAM will offload to CPU?
Yep. Technically, the other option is better (tell it to auto adjust / put as much as it can on GPU and then spill over onto CPU / RAM) The setting for that should be something like -ngl auto (in the latest versions of llama.cpp)
The NVIDIA RTX A4000 is nothing to sneeze at, 16GB or not - it's just...old, so support for it is middling. You should definitely try one of the MoE models on that. See if this helps:
https://www.youtube.com/watch?v=8F_5pdcD3HY
The setting he uses are in his first comment.
If I had to pull a number out of my butt, you should be able to triple his thru put (given that the A4000 is almost 3x the bandwidth, has actual tensor cores etc)
Incidentally, I can tell you that Qwen 3.6 35B-A3B, at Q4_K_M, --ctx 8192, runs at 6-8 tok/s on that CPU…because I tested it myself (llama.cpp, beellama and ik_llama), with llama.cpp being fastest overall.
Heh...digging around my .bat files, I found the launch params for that. Fucking ouch -
cd /d "%LLAMACUDA_DIR%" start "Qwen3.6-35B-A3B-UD-Q4_K_M" /min llama-server.exe ^ -m "%MODEL_PATH%" ^ -ngl 0 ^ --n-cpu-moe 35 ^ --no-mmap ^ --mlock ^ --cache-type-k q4_0 ^ --cache-type-v q4_0 ^ -c 8192 ^ --host 0.0.0.0 ^ --port %PORT%- No, not really. For a start, that CPU / Igpu could at best have Vulkan support...except the i7-8700 is AXV2...which is about 4 gen out of date, so have dicey vulkan support in Windows / a little better in linux.. Ask me how I know.
But even if you could, it would be slower than a real GPU for AI compute due to lack of tensors in the Igpu. You'd probably (definitely?) be better off just running straight on CPU in that case. And this is before even talking about different OS issues in windows / Linux...
Save yourself the pain; don't try to force inference via Igpu; you're gonna have a bad time.
It'd be dynamically allocated (generally speaking)...but see 1.
Sadly, it won't help so much as you think...because you are memory bandwidth bound. Though I suppose it does allow you to run larger models (slowly) on CPU...if you hate yourself :)
Incidentally, I can tell you that Qwen 3.6 35B-A3B, at Q4_K_M, --ctx 8192, runs at 6-8 tok/s on that CPU...because I tested it myself (llama.cpp, beellama and ik_llama), with llama.cpp being fastest overall.
If you already have a 16GB, DDR6 GPU, find something that plays nicely with it.
If you're asking about spill over or offloading a MoE, that should be automatic / with launch flags.
The very easy answer to this is "anything by Qwen", because they are the community darlings. But...is your use case specifically AI radio? Because that maybe speaks to something like ACE-Step 1.5 (...which is a Qwen 3 derivative)
Maybe ... but you'll then have to deal with split inference across 2 GPU (or I suppose run different things on different GPUs). Possible but...you'll have to look into orchestration.
They do, but much less than at peak. For example, if your GPU sucks down 250w at peak, it might idle at 100w. So if you have 2 GPUs + everything else, it might idle at 300w and peak near 650w. I suppose that's OK if you're not hammering it 24/7
I do hope people realise that 7th gen and earlier consoles are still fucking awesome. Yes, that's PS3/Xbox360/Wii/GBA/DS era.
Some of those still have lively homebrew scenes. Hell, the Wii just got a new (physical) release game.
Reject modernity. Embrace retro.
Over-write first, then deletes. Optionally, allows you to do download your comments first for archival.
Oh, that's so weird. How do I enable this in SmartTube?
Googz?
HOW DO I WATCH YOUR ADS IN SMART TUBE?
Do I need to use yt-dlp with TubeArchivist?
IS THAT WHAT WE SHOULD DO GOOGLE?
https://www.tubearchivist.com/
https://github.com/yuliskov/smarttube
Oh, and while I'm here -
It could be Qwen 3.5 397B-A17B (MoE) or Qwen 3.5 27B dense- both score comparably on the Artificial Analysis Intelligence Index (AAII). Lumo should come clean about exactly which models are being served at which quants; this surely isn't a state secret. The context window is obviously capped at 128K. I'd like to know if we're being served at least Q4_K_M - ideally better.
Given that GLM 5.2 has a 1M-token context window and Qwen 3.5 supports up to 1M via scaling (with 262K native), being served 128K makes quant quality all the more important. I'd like some assurance it's not IQ2_XS or similar.
I'm trialling Lumo AI and have paid for a month to test it. So far... results are mixed, trending towards good.
Issues:
- It defaults to an internal search tool over live search, which causes hallucinations from stale or incomplete content. This is significant.
- Runs out of context mid-chat (despite compaction), necessitating a page refresh to clear.
- The latest Android app (downloaded directly from F-Droid) is abysmally laggy (this should be priority 1)
- Cannot download generated images on Android.
- Can only search across 200 files from an attached Proton Drive folder; please increase this limit.
- On that: make it easier to attach / detach / update files from within Lumo. Adding to Files is not immediately updated on Lumos end, resulting in hallucinations against contents.
- Usage limits are opaque. It would be nice to have a rough sense of what "unlimited" means in practice (likely a ~5hr/day total message limit) so I can plan my work accordingly. I am aware of the image use limits (100/day for paid users) per announcement.
I'm trialling Lumo because its privacy stance is appealing (and I cannot run GLM 5.2 or 397B on-device), but the issues above are dampening my enthusiasm. They're all fixable. So please, Proton team, take this as critical (paid) user feedback. @ProtonPrivacy@lemmy.world
Hmm. Not sure I agree with "just mark the parts that are AI generated" because that obfuscates the parts that were human made, skewing perception towards "it's all AI gen".
Require the full accounting - human, clanker, level.
- Design - Human
- Implementation - Pair
- Testing - Assist
- Documentation - Human
- Review - Human
- Deployment - Human
Reads differently to
- Implementation - Pair
- Testing - Assist
4/6 human vs ?? / Human is a different trust signal (which is what this is actually about, right?)
PS: I'm a fan of acronyms, so how about "show us the STACK or show us the DIRTY"
- Spec (Design)
- Testing
- Assembly (Implementation)
- Checks (Review)
- Knowledge (Documentation)
Or
- Design
- Implementation
- Review
- Testing
- Yeet (deployment)
Those are rookie numbers. They need to pump that up. $10K a console, and you need to scan your retina to use it.
Ya know... for a luxury good with literally dozens of other, better options... "Make it impossible to buy our stuff" is a dumb hill to die on.
Oh well.
laughs in 40 years of non PlayStation games
Basically, yes
See