The main problem is when following instructions for command line tools. They might figure out how to use dnf instead of apt, but the extra layers required for ostree are not very friendly. There are a ton of potential frustrations in this area, especially with GPU stuff or hobbyist hardware like Arduino where kernel stuff is needed in userland. At least as of nearly 3 years ago, the documentation in this area sucks. I was on Silverblue for a few years and managed to get through the frustrations due to intermediate experience level. I found toolbox useless compared to distrobox. But using this with something like Arduino was annoying at best. The needed dependencies expected by whatever stuff I wanted to install was usually a big mystery with near useless error failure messages and names of packages and libraries totally unrelated to the package naming in DNF. When updating the base OS, stuff built in these containers is totally useless because I could not update the containers to the new OS image. Playing around with Flash Forth on a microcontroller was even worse. I ended up layering a bunch of stuff on the host because the containers were just not working. When I got an Nvidia machine, I went to Fedora Workstation and have had far fewer issues and frustrations. SB wasn’t bad, but it is a pain to use these if you need kernel level access. Just my $0.02. I was actually on SB for ~2-3 years.
𞋴𝛂𝛋𝛆
- 6 Posts
- 78 Comments
llama.cpp is at the core of almost all offline, open weights models. The server it creates is Open AI API compatible. Oobabooga Textgen WebUI is more user GUI oriented but based on llama.cpp. Oobabooga has the setup for loading models with a split workload between the CPU and GPU which makes larger gguf quantized models possible to run. Llama.cpp, has this feature, Oobabooga implements it. The model loading settings and softmax sampling settings take some trial and error to dial in well. It helps if you have a way of monitoring GPU memory usage in real time. Like I use a script that appends my terminal window title bar with GPU memory usage until inference time.
Ollama is another common project people use for offline open weights models, and it also runs on top of llama.cpp. It is a lot easier to get started in some instances and several projects use Ollama as a baseline for “Hello World!” type stuff. It has pretty good model loading and softmax settings without any fuss, but it does this at the expense of only running on GPU or CPU but never both in a split workload. This may seem great at first, but if you never experience running much larger quantized models in the 30B-140B range, you are unlikely to have success or a positive experience overall. The much smaller models in the 4B-14B range are all that are likely to run fast enough on your hardware AND completely load in your GPU memory if you only have 8GB-24GB. Most of the newer models are actually Mixture of Experts architectures. This means it is like loading ~7 models initially, but then only inferencing two of them at any one time. All you need is the system memory or the Deepspeed package (uses disk drive for excess space required) to load these larger models. Larger quantized models are much much smarter and more capable. You also need llama.cpp if you want to use function calling for agentic behaviors. Look into the agentic API and pull history in this area of llama.cpp before selecting what models to test in depth.
Huggingface is the goto website for sharing and sourcing models. That is heavily integrated with GitHub, so it is probably as toxic long term, but I do not know of a real FOSS alternative for that one. Hosting models is massive I/O for a server.
3·4 months ago
I have never used or cared about this W11. It has never seen the internet. I only keep it around for my keyboard’s RGB controller app if I ever need it. So I have no clue if this is everything or whatnot, but that is a screenshot of my access to the windows file system from within the file manager of Fedora. That is a dual boot partition. Fedora is particularly good at coexisting with a dual boot partition.
Just be aware that W11 is secure boot only.
There is a lot of ambiguous nonsense about this subject by people that lack a fundamental understanding of secure boot. Secure Boot, is not supported by Linux at all. It is part of systems distros build outside of the kernel. These are different for various distros. Fedora does it best IMO, but Ubuntu has an advanced system too. Gentoo has tutorial information about how to setup the system properly yourself.
The US government also has a handy PDF about setting up secure boot properly. This subject is somewhat complicated by the fact the UEFI bootloader graphical interface standard is only a reference implementation, with no guarantee that it is fully implemented, (especially the case in consumer grade hardware). Last I checked, Gentoo has the only tutorial guide about how to use an application called Keytool to boot directly into the UEFI system, bypassing the GUI implemented on your hardware, and where you are able to set your own keys manually.
If you choose to try this, some guides will suggest using a better encryption key than the default. The worst that can happen is that the new keys will get rejected and a default will be refreshed. It may seem like your system does not support custom keys. Be sure to try again with the default for UEFI in your bootloader GUI implementation. If it still does not work, you must use Keytool.
The TPM module is a small physical hardware chip. Inside there is a register that has a secret hardware encryption key hard coded. This secret key is never accessible in software. Instead, this key is used to encrypt new keys, and hash against those keys to verify that whatever software package is untampered with, and to decrypt information outside of the rest of the system using Direct Memory Access (DMA), as in DRAM/system memory. This effectively means some piece of software is able to create secure connections to the outside world using encrypted communications that cannot be read by anything else running on your system.
As a more tangible example, Google Pixel phones are the only ones with a TPM chip. This TPM chip is how and why Graphene OS exists. They leverage the TPM chip to encrypt the device operating system that can be verified, and they create the secure encrypted communication path to manage Over The Air software updates automatically.
There are multiple Keys in your UEFI bootloader on your computer. The main key is by the hardware manufacturer. Anyone with this key is able to change all software from UEFI down in your device. These occasionally get leaked or compromised too, and often the issue is never resolved. It is up to you to monitor and update… - as insane as it sounds.
The next level key below, is the package key for an operating system. It cannot alter UEFI software, but does control anything that boots after. This is typically where the Microsoft key is the default. It means they effectively control what operating system boots. Microsoft has issued what are called shim keys to Ubuntu and Fedora. Last I heard, these keys expired in October 2025 and had to be refreshed or may not have been reissued by M$. This shim was like a pass for these two distros to work under the M$ PKey. In other words, vanilla Ubuntu and Fedora Workstation could just work with Secure Boot enabled.
All issues in this space have nothing to do with where you put the operating systems on your drives. Stating nonsense about dual booting a partition is the stupid ambiguous misinformation that causes all of the problems. It is irrelevant where the operating systems are placed. Your specific bootloader implementation may be optimised to boot faster by jumping into the first one it finds. That is not the correct way for secure boot to work. It is supposed to check for any bootable code and deplete anything without a signed encryption key. People that do not understand this system, are playing a game of Russian Roulette. There one drive may get registered first in UEFI 99% of the time due to physical hardware PCB design and layout. That one time some random power quality issue shows up due to a power transient or whatnot, suddenly their OS boot entry is deleted.
The main key, and package keys are the encryption key owners of your hardware. People can literally use these to log into your machine if they have access to these keys. They can install or remove software from this interface. You have the right to take ownership of your machine by setting these yourself. You can set the main key, then you can use the Microsoft system online to get a new package key to run W10 w/SB or W11. You can sign any distro or other bootable code with your main key. Other than the issue of one of the default keys from the manufacturer or Microsoft getting compromised, I think the only vulnerabilities that secure boot protects against are physical access based attacks in terms of 3rd party issues. The system places a lot of trust in the manufacturer and Microsoft, and they are the owners of the hardware that are able to lock you out of, surveil, or theoretically exploit you with stalkerware. In practice, these connections are still using DNS on your network. If you have not disabled or blocked ECH like cloudflare-ech.com, I believe it is possible for a server to make an ECH connection and then create a side channel connection that would not show up on your network at all. Theoretically, I believe Microsoft could use their PKey on your hardware to connect to your hardware through ECH after your machine connects to any of their infrastructure.
Then the TMP chip becomes insidious and has the potential to create a surveillance state, as it can be used to further encrypt communications. The underlying hardware in all modern computers has another secret operating system too, so it does not need to cross your machine. For Intel, this system is call the Management Engine. In AMD it is the Platform Security Processor. In ARM it is called TrustZone.
Anyways, all of that is why it is why the Linux kernel does not directly support secure boot, the broader machinery, and the abstracted broader implications of why it matters.
I have a dual boot w11 partition on the same drive with secure boot and have had this for the last 2 years without ever having an issue. It is practically required to do this if you want to run CUDA stuff. I recommend owning your own hardware whenever possible.
Any UEFI secure boot enabled distro will remove all boot entries without a valid package key or a shim to a valid key.
Glad you got it working.
10·5 months agoPipe Pipe is better than Newpipe. I use F-droid’s VLC front end for local music because the built in android back end is VLC. For everything else, in browser
The UEFI boot system is tricky and you need to get along with Secure Boot to do this. Secure Boot is outside of the Linux kernel. Both Fedora and Ubuntu have systems for this. Fedora uses the Anaconda system and I believe they do it best. I have had a W11 partition for 2 years and never used it once. It can’t even get on the internet with my firewall setup, but it is there and never had any issues the 3 times I logged into it.
I think all of the Fedora systems support the shim key and secure boot but I know Workstation does. For Ubuntu I think it is just the regular vanilla Ubuntu desktop that the shim supports. This may be somewhat sketchy with Nvidia or maybe not. Nvidia “”““open sourced””“” their kernel code but the actual nvcc compiler required to build the binaries is still proprietary crap.
I have a 3080Ti gaming laptop. It isn’t half bad with 16 GB of video RAM from all the way back in 2021. Nvidia is artificially holding back the vram because of monopoly nonsense. The new stuff has very little real consumer value as a result, at least with AI stuff I run. The hardware is a little faster, but more vram is absolutely critical and new stuff that is the same or worse than what I have from 3 generations and nearly 5 years ago is ridiculous.
The battery life blows and the GPU likely won’t even work on battery. It will get donkey balls hot with AI workloads, especially any kind of image gen. This results in lots of thermal throttling. All AI packages run as servers on your network. If you are thinking along these lines if running your own models, get a tower and run the thing remotely.
I manage, and need the ergonomics for physical disability reasons, but I still would prefer to have a separate tower to run models from.
Anyways, you can sign your own UEFI keys to use any distro, but this can be daunting for some people. The US defense department has a good PDF guide on setting your own keys. The UEFI bootloader for the machine may not have all key signing features implemented. There is a way to boot into UEFI directly and set the keys manually but this is not easy to find great guides on how to do it step by step. Gentoo has a tutorial on this, but it assumes a high level of competency.
Other than signing your own keys, the shim keys mentioned are special keys signed by Microsoft for the principal maintainer of the distro. These slide under the Microsoft key to keep secure boot enabled.
If you boot any secure boot enabled OS, the bootloader is required to delete any bootable unsigned code it finds. It does not matter if it is a shimmed Fedora or W11. If you have any other OS present in the boot list, it should be deleted. W11 is SB only, and this is where the real issues arise.
Are you insane? Debian is a base distro like any other and runs more hardware than any other. It has all of the bootstrapping tools to get hardware working.
Canonical is a server company and Ubuntu server is literally the product.
Arch is absolute garbage for most users unless you have a CS degree or you have entirely too much time on your hands and don’t mind an OS as your life project. Arch abhors tutorial content in all documentation and therefore dumps users into a rabbit hole regularly. Pacman is the worst package manager as it will actively break a system and present the user with the dumbest of choices at random because the maintainers are ultimately sadistic and lackadaisical. Arch is nearly identical to Gentoo with Arch binaries often based on Gentoo builds, yet Gentoo provides relevant instruction and documentation with any changes that require user intervention and does so at a responsible and ethical level that shows kindness, respect, and consideration completely absent from Arch. Arch is a troll by trolls for trolls. I’m more than capable of running it now, but I would never bother with such inconsiderate behavior.
deleted by creator
Fedora’s Anaconda system makes UEFI secure boot easy and ships with SELinux integrated but set to permissive by default. Their built in network filtering tools are pretty easy but I still just use OpenWRT on a separate device. Silverblue was nice for a few years but I switched to Workstation for a machine with Nvidia hw.
You lose the I/O and power efficiency is no comparison. You can get better power efficiency and sometimes some I/O with an old router and OpenWRT, but you’ll be in the class of a Beagle Bone and a much harder learning curve. I’ve never managed to get a sensor or peripheral working on some old laptop’s SPI or I2C buses like how easy it is on a Rπ.
1·11 months agodeleted by creator
I use
alias man='batman --pager="less -RF"for colorized manpages on my workstation, but I’ll save this for sure. It might be handy on smaller embedded systems.I wonder why colorizing manpages like this is not default in most distros. That seems like an obvious thing to configure for end user’s quality of life.
Why are you confrontational? I’m just casually tossing out ideas and learning. Of course I understand what you are saying. However, busybox covers the core of a POSIX system and with the size constraints, it is likely standardising something like this. On Gentoo, such a change might be more straight forward instead of some sloppy hack with a wrapper.
I imagine you must be good at memorizing a lot of information. I am not. I am good at abstraction and must explore in abstraction to understand heuristically. I understand heuristic connections better than most people. Neither method is better or worse. Being toxic about interchanges of information is useless nonsense. I know far more than I let on, but I’m well aware that I am a jack of all trades and expert of none. All the projects don’t matter relative to those that are used the most. If most projects can be colorized, it will motivate others to fall in line or prompt rewrites assuming such a change was popular. Colorized manpages and help pages should be standard and should have been a decade ago. No one is using an IDE without syntax highlighting. The terminal is an extension of the abstracted language of Linux. Without universal syntax highlighting for new users in these spaces, Linux is presenting an outdated language format ripe for deprecation. These details have long term consequences.
I wonder if Busybox or similar rewrites contain standardization that could be leveraged.
I don’t mind the idea of a wrapper it is just that most of the time, I’m looking at the last command, backspacing and then adding
--help. After thinking about it, I will likely go the wrapper route, but add arguments that use the last command in terminal history automatically so that typinghelp-with no args runs a--helpflag on that last command,2::5would add additional flags or arguments from the last command before--helpandhelp-with any other args calls those instead of using history.
At this point, someone has to have already made a prettier shell or terminal that is configured like this by default. Hideous 1950s monocolor --help output can’t be a novel issue in 2025.
deleted by creator
There has to be a hook somewhere for every command that executes. I’m not sure, but something in the chain after using
set -xthen running any terminal command likely is on the right path to doing this. (If you tryset -x, you can turn it off withset +x).set -ooptions are another I’m not very familiar with but might be related.
Probably nothing helpful as you are already way past my understanding. Maybe look at the Darktable documentation or even the “green lantern” stuff (IIRC the name). GL or (something) Lantern is/was an open source software for Canon cameras that breaks out all DSLR features on nearly any Canon camera.
Nearly a decade ago, I had a makeshift product photography studio and messed with Macbeth color charts and profiles matched to a monitor. The tutorial guides I followed were from these two projects IIRC. GL.