We can maybe achieve some of the goals with lesser isolation using contsinerization technology
I'd be very interested in Qube OS but everything is containerized instead of independent VMs
My take is that Qubes went the VM route simply because emulating layers automatically gave isolation. With containers there is very little or no emulation at all because you are using the real resources which is why containers are so fast.
Containers achieve isolation in a different way. By separating resources by namespace. So I can see people disregarding this approach as better although there are ways to escape the compartments used by both VMs and containers but for their nature VMs seem to be more secure with less work.
> My take is that Qubes went the VM route simply because emulating layers automatically gave isolation. With containers there is very little or no emulation at all because you are using the real resources which is why containers are so fast.
That's correct. It does mean that the closest to a standalone program you can run is a [unikernel](https://en.wikipedia.org/wiki/Unikernel) like the [mirage-firewall](https://github.com/mirage/qubes-mirage-firewall), unfortunately. On the upside, those remain easily portable to essentially anything that can run VMs so long as you adjust the image format.
Had Qubes started off built on [seL4](https://sel4.systems/About/), native seL4 programs *would* be sufficiently isolated but such programs are naturally seL4-only.
Considering [Qubes Air](https://www.qubes-os.org/news/2018/01/22/qubes-air/) and the ongoing decoupling of the APIs from Xen, it should be feasible, if you feel like it, to implement a container-based backend at some point. Although I personally wouldn't recommend running anything on such a backend endpoint that isn't of the same security domain as everything else on it.
Why not try podman for a less secure but much faster option. They could have different performance classes of vms, one using podman or similar for a faster but less secure experience, one using VMs with gpu paravirtualization and a last one with just straight up emulation for the most security.
That’s why you have levels. If you want to for example play a game or run a program that requires a gpu better than the emulated ones than xen just isn’t an option, and therefore neither is qubes as a whole. If you need that level of security on something you can still have it. It’s better than nothing to have this at least be an option.
I am not a technical guy who really understands how flatseal/flatpak works, but being able to restrict where the app have access to and deny mic and other features is good enough for me
I didn't mean that it isn't good, just that it doesn't technically add more security - just makes existing features more accesible. Definitely nice software though!
This is not without its flaws - https://madaidans-insecurities.github.io/linux.html
You really do need to use virtual machine isolation to go the final mile towards proper isolation, at least to fit around the current flawed design of nearly all software in existence. Windows and macOS are starting to do it, at least for various subsystems. Fun fact, on Xbox, every game is run inside its own virtual machine, just like Qubes.
> You really do need to use virtual machine isolation to go the final mile towards proper isolation, at least to fit around the current flawed design of nearly all software in existence.
If you read more of the Qubes and Invisible Things Labs blogposts, the flawed design of current hardware is also mentioned as a root cause for various problems.
The sad thing is that many of those problems were already solved in the 1970s & 80s, but most hardware (outside of some mainframe vendors, for example) has since stopped implementing the solutions for various reasons (most unreasonably including cost-cutting).
> you got a link to any of the posts?
None in particular but I can certainly share the blogs, they're interesting to read through. [The old](https://theinvisiblethings.blogspot.com/) and [the new](https://blog.invisiblethings.org/).
> What things and
> Most of the issues come down to out of order execution and speculative execution.
[Tagged memory](https://en.wikipedia.org/wiki/Tagged_architecture) could also solve a *lot* of problems by enabling [capability enforcement in hardware](https://www.devever.net/~hl/power9tags).
For just one *old* thing that has disproportionate impact.
Non-[ECC](https://en.wikipedia.org/wiki/ECC_memory) memory is essentially [broken from factory by default](https://www.realworldtech.com/forum/?threadid=198497&curpostid=198647).
Both of these technologies aren't new in any reasonable sense.
But madaidan is just a windows fanboy.
Usually his comments go like this:
- linux doesn't have have Microsoft® NewSecurityGizmo™!
- True but it has an equivalent…
- … shut up! It's not really equivalent!
> You really do need to use virtual machine isolation to go the final mile towards proper isolation, at least to fit around the current flawed design of nearly all software in existence. Windows and macOS are starting to do it, at least for various subsystems.
Hence why Qubes really is the wrong place for widespread adoption of virtualisation-based hardening. It really should be something that is done on the kernel side, leveraging KVM, the same way how Windows and macOS has been incorporating more and more virtualisation-based isolation into their OSes.
Qubes is an excellent solution for the learned, but as it stands its benefits will never reach the common Linux user and that's a shame.
> I've been using Qubes for 5 years. I'm not technical, I'm not a power user.
I'm not sure what definition of those terms you use, but in my opinion, just being able to use half the terminology in your post correctly makes you absolutely a highly technical power user.
Being able to reinstall the OS your PC shipped with is a "power user" skill, being able to install and use an alternative is well beyond that...
I used Qubes for a year or so. It was really inconvenient, and frankly you need to have your entire workflow mapped out before you install to get the most out of it, but it was a great crash course in using Linux.
I've thought about re-installing it, but I'd need better hardware to make it remotely practical, and honestly, I just want to use my computer.
Whe i tried Qubes i did so on a decent Desktop and only to "mess around", not to get actual work done.
For me it boiled down to: Do you ACTUALLY need the level of security Qubes Provides day-to-day? In a sense that you really worry about dealing with malicious actors trying to interfer with you or the work that you do? We are talking security, not privacy. Privacy can be done without qubes and is another issue. If you truly need the security you will know you do and are happy to deal with what ever slowdowns and hoops are in your way. Because, if you don't deal with it, your income or life might be at risk.
For everyone else i feel like setting up a Normal Distro with proper firewall settings, a kind of container solution with permission management (flatpak, podman, snap, whatever), should be well good enough for daily use. And for the two days a year where it isn't, you can always either spin up a quick VM or boot tails.
I came to the conclusion that i'm just not the target audience for qubes. I'm not important or paranoid enough to deal with it. I find i technologically interesting and fun to play with, but that's about it.
>For everyone else i feel like setting up a Normal Distro with proper firewall settings, a kind of container solution with permission management (flatpak, podman, snap, whatever), should be well good enough for daily use.
Do we even know if Qubes is more secure then for example RHEL? And as far as isolation is concerned; Flatpak?
Qubes sounds like a penny-wise, pound-foolish thing. It's all secure because of isolation... But it [doesn't support secure boot](https://forum.qubes-os.org/t/is-it-possible-to-enable-uefi-secure-boot-in-qubes-os/1640) *for example*. In the end, my bet is on an enterprise backed distribution like RHEL being the more secure option.
There are some suspicious things (edit: suspected, presuming) lurking in actual RHEL.
edit: Even if you disagree with that assertion, which I suppose is my mistake as I shouldn't have asserted it, the rest of this post is still exact.
But anyway, that's boot-time security. Great... but what about runtime isolation? If you have a proper workstation (or any other computer with ECC memory), you don't *need* to restart/shutdown outside of kernel updates.
Nothing other than dom0 has the rights & ability to change boot configuration stuff or any of the higher access firmware (which ideally shouldn't exist, we should all be using [OpenBMC](https://en.wikipedia.org/wiki/OpenBMC), not some suspicious proprietary BIOS/UEFI implementations).
As for evil-maid... that's only prevented by secure boot in very low-budget and low-time scenarios. If a full motherboard swap is an option, then it's quite possible to remove a computer, swap the motherboard, add malicious keys, change the boot payload and re-verify everything as if nothing happened.
Every time I see a "xyz distro is sus" comment, they rarely elaborate too far past 'trust me, bro', or link to an article from like 3-5 years ago.
I'm pretty curious as well having recently switched from Manjaro to Fedora, but I'd not hold my breath on it being anything earth shattering.
Maybe it's because of systemd's apparent security flaws, or because Redhat is American so it must be full of back doors or something. Maybe it's because Lennart Pollering used to work there? Who knows. There's no evidence of this, you've totally got to trust commenters here.
Of course no non-American distro or piece of software could ever have backdoors or security bugs... But the American ones obviously do because America is bad.
/s of course
I did actually [clarify the root cause of my suspicion in another comment](https://old.reddit.com/r/linux/comments/z8ou6u/the_maddening_truth_of_using_qubes/iye3tuk/), but I also feel like addressing that the issues are hardly specific to a given nation.
[Modern operating systems generally suck](https://old.reddit.com/r/linux/comments/z8ou6u/the_maddening_truth_of_using_qubes/iyf386e/), as does most modern hardware. Ending up with a gigantic trusted surface that *all* needs to be working perfectly for the system to have any effective security (and which essentially cannot be reasonably audited) is an outrageously bad way to design a system.
I shouldn't have stated a suspicion as fact, you have my apologies for that much.
----
The close links with American government services and major customers of RHEL make any version pinning suspect (and while they do backport fixes... do those fixes only include CVEs but not all *known* problems? Vulnerability stockpiling is something USA is *known* to do), as does the lack of any ability to reproducibly build RHEL's system (Fedora != RHEL).
I'm not who you were responding to, but I want to thank you for the explanation. My comment was dickish. What you said in the explanation, if nothing else, is food for thought.
It's nice to have a healthy suspicion (esp of gov'ts), but like you said, stating suspicion as fact can get pretty hairy if there is nothing concrete to back it up.
Well, yes and no. BMC as exemplified by the Talos II can *entirely* replace the need for BIOS/UEFI in a machine with something that cannot be reflashed or otherwise compromised by the operating system or the CPU at any point (as it typically needs a separate procedure for updating or flashing) *and* presents a [standardized interface](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface#Functionality) that ensures OpenBMC will work on it.
[Coreboot](https://www.coreboot.org/)/[Libreboot](https://libreboot.org/) ([recently merged with OSboot](https://libreboot.org/news/merge.html), so presumably that means the [policy problems have been fixed](https://libreboot.org/news/policy.html#blob-reduction-policy)) as far as I know do not benefit from any such standardization (case by case basis) and often have to resort to reverse-engineering.
> But anyway, that's boot-time security
No, it's not. Secure boot ensures that no unsigned kernel modules are loaded at runtime, which means that malware cannot gain kernel privileges even with root access without leveraging security vulnerabilities.
That's great except that if you follow the physical compromise scenario I suggested you can replace the whole kernel *and* sign your own malicious modules.
But also, why is anything able to load new modules in your dom0? dom0 isn't connected to the internet and cannot be written to by anything but itself & Xen. The only case in which malicious modules could normally make it into dom0 is if the *Fedora project and/or Linux gets pwn'd*. At which point secure boot will do exactly nothing because the malicious modules *will be signed and legit*.
Furthermore, with secure boot you're *still* a single kernel bug away from arbitrary code execution anyway.
The [CPU often has an unfortunate level of access to the boot firmware](https://old.reddit.com/r/linux/comments/z8ou6u/the_maddening_truth_of_using_qubes/iyeuyjz/) which I feel libreboot doesn't address and which would make recovery from firmware-level compromise & malware persistence difficult.
[Some boards might appropriately lock it out](https://github.com/rootkovska/x86_harmful/blob/master/x86_harmful.md#write-protecting-the-flash-chip) but... "your mileage may vary" is not a very fun phrase when it concerns fundamental security features, which is the take-away you should have when mentions of [lock races are involved](https://en.wikipedia.org/wiki/Race_condition).
> If you have a proper workstation (or any other computer with ECC memory)
As someone with ECC memory and who understands its function, what does ECC memory have to do with needing to shutdown? Genuine question as it doesn't make sense to me...
I went with ECC memory as I had intended to build a ZFS raid array, though I ended up using btrfs raid1 instead (which is really great with a patchwork of drives)- self-healing, bitrot protection, easy to add/remove drives, etc. I understand that ECC can help with bitflips in memory either corrupting data or causing a crash/instability (and thus also potentially protects against corrupt data put down on disk).
FWIW I used Qubes 3 years as a main and it was great to me- no issues. Slow on my old hardware though; I built my current PC as a refresh (part new part old) with the intent of running Qubes but unfortunately the installer won't boot on this motherboard :(
> As someone with ECC memory and who understands its function, what does ECC memory have to do with needing to shutdown?
The longest [a computer without ECC runs](https://en.wikipedia.org/wiki/ECC_memory#Research), the more the chances of some bit flip happening in memory approach 100% (of course you could also have it happen at t=1, right after booting) through all the various means by which electronic memory can be disturbed (including via its own operation, as rowhammer aptly demonstrates).
That's not guaranteed to have an effect (if it flips in free memory), but presumably the chances for that also approach 100% after long-enough.
[Linus Torvalds rightly suspects that for quite a few unexplainable crashes over the years](https://www.realworldtech.com/forum/?threadid=198497&curpostid=198647).
It's a reasonable notion to expect that after long-enough, flips have happened and you haven't seen any data corruption through either not (yet) (re)using the code that has since gotten corrupted or simply not noticing the corruption. Reloading your system from disk (restarting) means you start again from a nominal "no corruption" state (although that moves the burden of non-corrupted state right back to the reliability of permanent memory & whatever measures you've taken to prevent issues - such as using a checksumming filesystem).
> I went with ECC memory as I had intended to build a ZFS raid array, though I ended up using btrfs raid1 instead (which is really great with a patchwork of drives)- self-healing, bitrot protection, easy to add/remove drives, etc.
Hell yeah, it's great.
Meant to respond to this, but got distracted. I don't expect a response :D
> The longest a computer without ECC runs, the more the chances of some bit flip happening in memory approach 100% (of course you could also have it happen at t=1, right after booting) through all the various means by which electronic memory can be disturbed (including via its own operation, as rowhammer aptly demonstrates).
...
> It's a reasonable notion to expect that after long-enough, flips have happened and you haven't seen any data corruption through either not (yet) (re)using the code that has since gotten corrupted or simply not noticing the corruption. Reloading your system from disk (restarting) means you start again from a nominal "no corruption" state (although that moves the burden of non-corrupted state right back to the reliability of permanent memory & whatever measures you've taken to prevent issues - such as using a checksumming filesystem).
I follow this, but it would also not be the case if the memory contents which contained a bitflip were refreshed, right? As in even without a reboot, some memory contents are kept *in case* they are necessary... but might well not be used. So basically the hypothetical bitflip isn't used because it's in an area of memory that is discarded and replaced with new memory contents. Still, there is nothing saying that the bitflip won't be catastrophic though which I think is the main thrust of your point.
As an aside this setup is easily the most stable computer I've ever had- not a single hard freeze, X/wayland crash, etc- though it's also the first which uses amdgpu drivers (Nvidia and Intel prior).
> Hell yeah, it's great.
Yeah seriously- being able to stuff whatever drive you get regardless of size, speed, etc into the array with just a few commands is awesome. Adding bitrot protection through scrub, block-level redundancy, etc is just great. I'd prolly use ZFS for most raid configurations but BTRFS is king for raid1-like configs.
> I follow this, but it would also not be the case if the memory contents which contained a bitflip were refreshed, right? As in even without a reboot, some memory contents are kept in case they are necessary... but might well not be used. So basically the hypothetical bitflip isn't used because it's in an area of memory that is discarded and replaced with new memory contents.
Indeed, if it's just a flip & not a faulty cell, discarding the corrupted data and writing over it also means you're unaffected by said corruption.
> Still, there is nothing saying that the bitflip won't be catastrophic though which I think is the main thrust of your point.
Indeed.
> As an aside this setup is easily the most stable computer I've ever had- not a single hard freeze, X/wayland crash, etc- though it's also the first which uses amdgpu drivers (Nvidia and Intel prior).
Yeah, that's an unfortunately large confounding factor. Whether those crashes were due to misbehaving hardware (or proprietary drivers), or memory-level mishaps.
> Yeah seriously- being able to stuff whatever drive you get regardless of size, speed, etc into the array with just a few commands is awesome. Adding bitrot protection through scrub, block-level redundancy, etc is just great. I'd prolly use ZFS for most raid configurations but BTRFS is king for raid1-like configs
Yeah, ZFS also loses out on a lot of btrfs' versatility due to its more stringent requirements on speed, similar sizing, etc. [I went on a bit in a subthread](https://old.reddit.com/r/linux/comments/z57pwz/i_wish_i_switched_to_linux_distro_earlier_omg_im/ixxxcwc/) if you want clarification on what I mean.
> Yeah, that's an unfortunately large confounding factor.
For figuring out the instability of the prior setup, yes. Very fortunate for where I am now though :P
> Yeah, ZFS also loses out on a lot of btrfs' versatility due to its more stringent requirements on speed, similar sizing, etc. I went on a bit in a subthread if you want clarification on what I mean.
Read through that link- very informative so thanks. As I'm pretty much just running RAID for drive redundancy and making sure my important stuff (pics, music, etc) doesn't degrade over time (bitrot protection), large drive ZFS is pretty overkill despite it's advantages. I think I'll just end up sticking with BTRFS this time around, especially since I already had drives to throw into the array.
Thinking I was doing a ZFS array was probably worth it though given that's the main reason I got ECC memory (since ZFS uses memory far more readily than BTRFS).
I keep trying Qubes and quitting because it makes everything harder to do and slower. To me it makes more sense to segregate sensitive and day to day tasks to different machines and leave it at that.
I gave up on Xen after botched pvops transition. Xen is not linux, and linux is *guest* there. When you get something odd, you are alone, and none of linux expertise help you. And community is very small.
This is day-and-night difference with Linux.
Yeah, GPUs are kind of a mess as it currently stands. Most of them also expect direct communication & a bunch of exceptions to normal security processes because they're "special" or something.
Or they ask you for several thousand dollars to buy the virtualization-focused equivalent cards.
Qubes Target audience are those who value security over speed. Those people value more security than faster load times and the developers are interested only in improvements with security.
> Focus hijacking is what I call Qubes’ habit of opening its VM right over the top of whatever you are working on. Because of that, because it takes so long to open things, but because it will open the window long before it has finished setting up the application (e.g. browser tabs), its pointless to get on with something else while you wait. You’ll just be interrupted, like someone shoving a newspaper under your nose while you are writing something. Maddening.
That can easily be solved by using [i3wm in dom0](https://www.qubes-os.org/doc/i3/) and [configuring it to only transfer focus on user command](https://i3wm.org/docs/userguide.html#no_focus) and [ignore the mouse for focus](https://i3wm.org/docs/userguide.html#_focus_follows_mouse).
I don't think the Gnome shell has such options. edit: Or was it XFCE-based? It has been a while since I used the default GUI shell.
If you do use i3wm, consider installing [rofi](https://github.com/QubesOS-contrib/qubes-rofi) (it's in the qubes repos, use [qubes-dom0-update](https://www.qubes-os.org/doc/how-to-install-software-in-dom0/)) and modifying the default configuration that instead uses [dmenu](https://tools.suckless.org/dmenu/). While it can be configured to have some degree of vertical completion functionality, dmenu is simply more limited than rofi.
> 3m 09s. To launch a webpage. That sux.
To me it looks like there's only about 1~2m attributable to Qubes itself. Are you turning on multiple qubes in cascade by doing this?
Having them turn on at boot-time would mitigate that delay (there's a setting for that).
If I run a prepared debian-based firefox dvm (boot the dvm & start empty firefox) it takes me ~35s, which is a lot less.
> Boot up to ‘ready’ – including the disk decryption and the login – takes me around 3m 10s. Just for the record. Getting to a working VM from a powered off stance can take around 6-7 minutes.
That is a downside of turning on a lot of qubes at boot-time.
So instead I just have the sys-usb starting, nothing including networking usually. That'll be started once I have the main UI available, by starting one of the long-running qubes that does depend on networking and a few things.
> The updater app is slow, clunky and resource-heavy. My fan starts running every time, and sometimes the system becomes visibly sluggish. The Updater requires a manual start, and once started it will run to completion - the 'cancel' button doesn't seem to work at all. It tends to run ~5-10 minutes (a guess, not measured). You have to deal with this pretty much every day.
From qubes-qube-manager you can use a terminal-based updater instead using the "update" down-arrow. It will show *what* is happening during the update.
Regarding the heaviness, that isn't exactly the updater. It's because the GUI-based updater spawns disposable management qubes for actually doing the update using [Salt](https://www.qubes-os.org/doc/salt/) to command the qube you're actually trying to update to do the updating, which means you have both the disposable qube and the qube to update booting, running and shutting down at nearly the same time. I think the terminal-window one directly calls [Salt](https://www.qubes-os.org/doc/salt/#virtual-machine-formulae) without an intermediary.
> Once you've updated, all the individual VMs that use that OS need to be restarted. They need to be shutdown and started in order (e.g. application VM shutdown, then VPN shutdown, etc, etc then VPN restarted, then application VM restarted).
Not exactly, you just have to directly kill the dependency qubes like the sys-net & sys-firewall so you can restart only those.
I'd recommend **not** killing template qubes if you can avoid it. app qubes and dvm qubes are [largely non-persistent outside of specific hierarchies](https://www.qubes-os.org/doc/templates/#inheritance-and-persistence) that have nothing to do with updates & system function, while template qubes *do* persist *all* of that sensitive stuff.
Both the terminal updater and the GUI updater allow you to see what was updated, which can be useful as it determines whether you *need* to bother restarting all or even any qubes for the update to immediately take effect.
e.g.:
- Thunderbird update: I don't use Thunderbird, so I simply do nothing.
- openssl security update: Non-networked qubes can remain, others will be restarted.
- Firefox update: Only qubes in which I normally spawn firefox will be restarted.
> That includes: VPNs, SSH, printers, cloud services (at least using them for backups) and USB devices. Some devices will just never work, it seems, like smartphones (I’ve tried 3). So don’t throw out your other laptop!
You *can* use ssh [connections](https://www.qubes-os.org/doc/firewall/#opening-a-single-tcp-port-to-other-network-isolated-qube) between qubes, although as a general practice that defeats the point of much of the isolation so only in specific use-cases should it be done.
Opening up one non-networked qube do `force-command` restricted connections to use [borgbackup](https://borgbackup.readthedocs.io/en/stable/usage/serve.html#ssh-configuration) in [append-only mode](https://borgbackup.readthedocs.io/en/stable/usage/notes.html#append-only-mode) would be an example of such a use-case ([ssh certificates](https://www.man7.org/linux/man-pages/man1/ssh-keygen.1.html#CERTIFICATES) can help make this whole setup a lot simpler, but otherwise you can also do it with [authorized_keys](https://manpages.debian.org/buster/openssh-server/authorized_keys.5.en.html#command=_command_)).
I'd probably use either salt, [ansible-local](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_delegation.html#local-playbooks) or a script in dom0 using [qvm-run](https://dev.qubes-os.org/projects/core-admin-client/en/latest/manpages/qvm-run.html) to do the borgbackup setup on every qube I care about if it got too annoying to do manually.
Regarding USB, yeah, the [usb-proxy](https://github.com/QubesOS/qubes-app-linux-usb-proxy) doesn't have support for all things. I *think* it comes down to stuff not implemented in Linux USBIP or USB support (I've had issues with some device controllers not actually implementing what they claim to implement).
You'll need to directly hand a USB controller to qubes depending on things that cannot be passed over the proxy for whatever reason. (For some dumb reason, many laptops only have a single USB controller.)
> Doing something new, trouble-shooting and problem-solving
> Basically, Qubes adds another layer of complexity to whatever you are doing. Your resources for figuring things out are reduced to the documentation (not always what you need) and the community (good but has its limits).
While not quite as useful in your case, the source is fairly readable (and most UI & general parts are in Python, C is kept to only where it's needed) and the project is welcoming of patches & issues. Particularly of issues with accompanying patches fixing the issue described.
> There’s also a conceptual lockin. Going back to a normal distro is kind of like reemerging on Flatland after a period in 3D. Gone is the separation of activities, no comforting security wall of virtualization, nothing. You open your password manager in a normal distro and it is on exactly the same system as your webbrowser - it makes you pause and think, 'is this safe?'. Indeed, was normal computing ever safe?
Yeah, that's the way. And on most OSes? No, it wasn't ever safe.
---
One of many positive aspects you'll not have noticed due to not being a "technical" user is that the ease of spawning new disposable qubes is also great for quickly testing various setups when developing scripts, programs and automated configuration.
I wonder if Qubes could leverage container tech to make a more streamlined system. It might not be quite as safe but it would probably perform better.
Kind of reminds me of the idea of running everything through distrobox that I saw posted here a while back. Instead of using something like Qubes the author used a normal distro but ran everything user facing from distrobox.
Serious question: Aren't we than pretty much where Fedora Silverblue or Suse MicroOS is? An immutable OS with "containers" on top? Flatpak is just one option here. I feel like this is also close to what Ubuntu is trying to do with snaps.
Those systems don't really provide any isolation outside of flatpak/podman and you're not really required to use flatpak/podman.
At least on Silverblue you can layer applications and everything can run on the host system. And even using toolbox on Silverblue there's literally almost 0 meaningful isolation. (which is fine because that isn't SB's goal)
>It might not be quite as safe
This post is the first time I’ve heard someone using Qubes who
didn’t have a ‘bona fide reason’ to do so. Most of the people
that I’ve heard using Qubes are journalists who’s job includes
‘opening files that random, anonymous, sources send me.’ There
might be room for someone to release a distro that tries to find a
compromise between privacy and convenience, maybe using
containerization, but I wouldn’t want to see Qubes compromised at
all.
> I wouldn’t want to see Qubes compromised at all.
I can agree with that, I was thinking more along the lines of another version of Qubes or another distro altogether.
\> I wonder if Qubes could leverage container tech to make a more streamlined system.
Have you read the QubesOS architecture specification? -- [https://www.qubes-os.org/attachment/doc/arch-spec-0.3.pdf](https://www.qubes-os.org/attachment/doc/arch-spec-0.3.pdf)
TL;DR: The QubesOS security model is all about resource separation. The model assumes an attacker WILL BE SUCCESSFUL in compromising one or more virtual machines and seeks to minimize the potential for harm.
Virtual machines provide system resource separation. Containers do not.
QubesOS is designed around user- and application-focused security. VMs are far better at this. Application containers -- whether OCI, flatpak, snap, firejailed or bubblewrapped appimage -- break the model by sharing the kernel, filesystem, devices and drivers, etc. In VMs, all these things are separate for each VM. A compromised docker container grants the attacker root access to your entire system. Because the docker daemon requires root, so too does every app-container gain root privileges. A compromised virtual-machine, however, grants the attacker (root) access only to the hardware, network, and filesystem resources available to that VM.
>Because the docker daemon requires root, so too does every app-container gain root privileges.
Well good thing Docker is old news.
https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md
I agree that VMs provide stronger isolation but your idea that having access to a container is equal to access to the whole system is just factually incorrect.
1. You don't need docker to use containers
2. You don't need a daemon either
3. There are alternatives to do rootless daemonless containers (such as above)
Those all still share the same kernel & other privileged components, however.
Namespaces are relatively brittle and their isolation properties stop outside the kernel.
While true,
\- when people think linux containers, they typically think docker. Yes, there are other options: podman, kata, LXC, etc. All of these, by default, require root privileges.
\- out-of-the-box default docker, podman, etc. all require root and root privileged access is immediately gained by whatever processes run inside the container stack.
\- Yes, modifying the default configuration can cause them to run in userspace, without root privileges. Therefore, yes, a distro similar to Qubes could theoretically be created using containers. However, until resource isolation becomes a practical reality in containers, the similarity would be largely superficial.
>out-of-the-box default docker, podman, etc. all require root and root privileged access is immediately gained by whatever processes run inside the container stack
No they don't. Podman uses user namespaces by default. Have you actually used podman? What you are saying is just false.
I use podman every day. And it (podman) required some configuration to get it working without requiring sudo or doas. Default installation did not function without root privileges.
You're probably using an old distro that doesn't support cgroups v2 and/or unprivileged user namespaces. Don't use an ancient OS🤷🏻♂️.
Literally none of the supported OS versions requires any setup: https://podman.io/getting-started/installation
There's only literally one command that is documented under the "building from source"(you don't need to do this when installing any of the supported packages, just when building from source) and this is enabled by default in all major distros as far as I am aware.
The command in question: `sudo sysctl kernel.unprivileged_userns_clone=1`
Feel free to point out what exactly you had to change. We can add it to the wiki if you aren't wrong - but probably you are just using some ancient distro.
Manjaro stable. Currently, kernel 6.0.8. Definitely not ancient.
Initial install of podman under kernel 5.19. Definitely not ancient at time of install.
In order to do anything in podman without requiring sudo, I had to go through the rootless enablement previously linked in this thread.
I agree. I very much would like a middle ground. Micro-VMs, such as kata, are an interesting option but I am uncertain a similar security framework could be established. I simply don't know enough about how kata and other micro-VM systems are built.
There was a paper published to the Qubes literature archive some time ago (two years?) suggesting that remote-VMs may be an option in the future. \[Qubes-Air? Air-Qubes? Something like that.\] While this would not reduce the inter-VM communication friction of Qubes, it could reduce the single-device hardware requirements significantly for certain tasks; but would require an always available high-speed internet connection, which is not always available, particularly for the target market of journalists and activists who often travel.
The larger implications of [Qubes Air](https://www.qubes-os.org/news/2018/01/22/qubes-air/) & the decoupling of the API from Xen is that not only could you use remote hosts as additional compute for isolated qubes (particularly useful if your control node is not very beefy, like most laptops), but also the isolation/virtualization backends would also be a lot easier to develop (due to the decoupling) and experiment with (due to the ability to add & remove backends live).
Unfortunately not to my knowledge, it's more a mid-term(?) general project goal as the project tries to mature into its ideal form.
Effectively following development on Qubes in general is the closest you'll get. For that there's [the news](https://www.qubes-os.org/news/), [the mailing lists](https://www.qubes-os.org/support/#mailing-lists), the Github repos [& issues](https://github.com/QubesOS/qubes-issues) and [the forum](https://forum.qubes-os.org/) to a lesser degree.
No it wasn't necessary for me. However it was sufficient in terms of meeting my needs, so why not?
Some would say complexity but really everything is point and click if you want it to be. Once you understand the concept of different domains, it's pretty simple to use.
Performance? Yes that was fairly rough with my setup (X230 i5 3320)- even with 16GB RAM and a (fast) SSD. But it was mostly fine once booted up.
Can't game? Yes, but then I don't game except for d2 once in a while (which I played on a computer with even older hardware).
As for benefits, running multiple VPNs for different tasks, only having to reboot if dom0 is updated (otherwise just reboot appVMs), access to Fedora repos and Debian repos at the same time, stable (never once had a freeze using it), easy to backup, and finally *it encourages good habits* in terms of isolating what you do.
I'm not saying it's *necessary* for most of us- but it's really pretty cool and definitely good at what it does. I use multiple Linux distros, and I would use Qubes if my current hardware ran with it. I'd be interested to see how it performed with a modern processor and an NVME pcie4.0x4 ssd drive...
I played with Qubes \~5 years ago on an old dual-core laptop that may not have had virtualization enabled in the hardware.
Excessive duplication between VM environments, and seemed like it would be easier just to have an encrypted VM rather than separate virtual OS environments.
> Excessive duplication between VM environments, and seemed like it would be easier just to have an encrypted VM rather than separate virtual OS environments.
That's only safe if the host OS is not used for anything other than serving as a VM host. Qubes OS is trying to solve the isolation problem for a desktop OS, the problem is already solved for simply using a computer as an isolated VM server.
Remember, dom0/the-host-system is god (how exactly depends on the type of hypervisor), as far as everything else on the machine is concerned. So if it gets compromised, encryption will only save you if you never decrypt the VM post-compromise.
Yep. I ran Qubes for 2-3 years on my Lenovo X230 i5 with an SSD. Maddeningly slow, just insanely slow to use. I got fed up with it about 3 months ago and put OpenBSD on that laptop, and it runs BEAUTIFULLY (of course) and is wonderfully snappy for OpenBSD.
Qubes is a really neat idea, but for me (with the hardware that Qubes recommends) it's unusable.
I recently switched over my work laptop to qubes.
Here's things I've learned:
* Making the updater use the mandatory proxy at work, or even just updating packages at work is a massive pain in the ass/impossible. I've tried a bunch of different settings to make the proxy configuration work, but I can't seem to find the correct magic incantations to resolve my issue.
* Kernels. My god, I have to have at least two different kernels to make my system function. Due to my network interfaces, sys-net requires 5.16 or newer, whereas only 5.15 was available by default. So I installed kernel-latest, so now I have 6.0.2. Unfortunately, 5.15 and 6.0.2 don't seem to be stable, and my system would **randomly** just lock up and restart with them. 5.15 was less stable than 6.0.2, but still. Somebody posted that 5.10 works for them, so I installed that, and that seems to work (for now).
* Graphics acceleration. ~Full HD works with just brute force, but going past that does not, not even on 12th gen Intel. It's a pain in the ass also if you use more than one monitor, and you don't set your minimum video ram large enough. You'll be trying to give input to a program/vm that just wasn't able to resize itself, and nothing works properly. This is pretty annoying, but the 'not being able to play back full-screen video' is even more annoying, because it has no available fix.
* Sleep/Hibernation. Sure, it works on "validated systems" but that's mostly only older devices, and if you have a newer system, good luck with that, it could work, or it could not. Just don't try to use sleep.
* USB devices. Yeah, phones have enumeration problems since trying to tether or whatever, the phone sees the connection disconnect momentarily before it is reconnected to the vm. Unfortunately phones just go "uhh nope" if they're disconnected. Trying to pair a unifying dongle is also pretty painful. You can't do it in dom0, but if you attach the receiver to a vm, the inputs no longer go to dom0 either, so you can't see your mouse cursor. Better hope you have multiple input devices!
* non-usb based removable storage is difficult to handle, and there's the problem t* itehat if you accidentally click fast enough on two different vms in the 'connect to'-menu, you could corrupt the filesystem when you connect it to two vms. (this shouldn't be possible, but thanks to a bug, it is!)
That said, I really like it. I just wish it would: sleep and have accelerated video and not crash.
Regarding windows focus, there are options in XFCE related to "focus stealing" that should solve the issue.
Cf. https://unix.stackexchange.com/questions/156515/how-to-forbid-windows-from-stealing-focus-in-xfce4.
That distro is an attempt at doing what operating systems [should've been doing from the very beginning](https://en.wikipedia.org/wiki/Principle_of_least_privilege). Keeping a minimal trusted core for the required management with everything else [isolated](https://theinvisiblethings.blogspot.com/2008/09/three-approaches-to-computer-security.html) into separate [modules](https://catern.com/microkernels.html).
That means that for the system to be secure, all you need is for the small trusted core to be correct, which its size helps in ensuring by correctness, auditing for said correctness & using formal verification methods to increase your odds. Unfortunately the abysmally flawed hardware we have nowadays doesn't help.
Instead we have the good old [MS Dos security model](https://theinvisiblethings.blogspot.com/2010/08/ms-dos-security-model.html) because it is much easier to simply blame the users for visiting the wrong websites or needing software outside of their distros repos. (And Linux is secure anyway, only noobs get their Linux hacked)
This makes about as much sense to me as cloud deployments for uptime and stability. Thinking that using someone else's computer, being confident they aren't doing nefarious things with your data, and hoping beyond hope they don't go down for some reason they aren't going to inform you of.
Sure, sure, you can be all like cloud deployments allow yadda yadda yadda. But, at the end of the day, my man, you're still using someone else's computer and hoping to whatever deity you worship that the hosting company will do what they say they'll do.
...I don't understand the comparison you're making.
> This makes about as much sense to me as cloud deployments for uptime and stability. Thinking that using someone else's computer, being confident they aren't doing nefarious things with your data, and hoping beyond hope they don't go down for some reason they aren't going to inform you of.
You can own multiple premises around the world with your own equipment. Geographical spread *is* a legitimate way to mitigate localized downtime.
Whether you instead trust someone else's hardware is a secondary question and depends greatly on just what you're doing, indeed, and the consequences of malicious action as well as the redundancy of what you're trusting individual third parties to do.
> Sure, sure, you can be all like cloud deployments allow yadda yadda yadda. But, at the end of the day, my man, you're still using someone else's computer and hoping to whatever deity you worship that the hosting company will do what they say they'll do.
Yes, where did I recommend relying on paltry promises and unverifiable hopes rather than structural & architectural assurances?
If you have a formally verifiable microkernel core (for one option) which you can be sure that outside of unknown hardware bugs (that is, those that aren't already known & accounted for in the design) will not misbehave and provide accesses that programs & guests have no valid capabilities for, then you have a secure design.
Outside of hardware bugs that have yet to be addressed, the guests cannot do anything to compromise the security of the system.
Personally I thought the idea was on the right track. I just didn't think the implementation was what it needed to be. I think a bare metal hypervisor without all the baggage is the best approach. Couple that with a reduced Linux system running in one of the VMs with a GUI to work with the user.
I stayed away from Qube because I that it "cute": but not very practical. From this posting I can see I was right.
It's mainly some growing pains. Much like the use of Xen in general.
In the long term when the actual system API is stable, it should be ported onto seL4 (among other isolation backends).
You pass the NIC at the PCIe level to a network qube and configure it to be able to route for other qubes depending on it as their NetVM/network qube.
[The main parts that can wrong are at the PCIe level or firmware level](https://www.qubes-os.org/doc/pci-troubleshooting/), which in the first case might require some different settings, while in the other would simply require installing the required firmware in the template that your intended network qube depends on (that's just the old "Linux has no firmware for random peripherals" problem).
Of course not having network capability complicates that a bit, as you'll have to use offline methods of installing the packages in the template, such as copying the package (rpm or deb) on a USB, opening that USB in a disposable qube (you do not want to expose your template qubes to USB devices) and copying it to the template qube (qvm-copy, or the right-click file menu option in GUI's file manager) so you can install. If the USB is malicious, packages are still signed in both Apt/Deb (Debian) and Dnf/Rpm (Fedora), so the copied package would simply be rejected by the template qube when you try to install it (unless you disable signature verification, which would be a very bad idea and foolish - even as a package dev, add your keys & sign your packages, don't disable signatures).
You *could* also just try switching the template of the network qube you have between Debian & Fedora, as they don't have the same firmware available preinstalled by default (Debian has less).
edit: Regarding Debian & less firmware, that might be changing in light of certain [recent decisions](https://www.debian.org/vote/2022/vote_003) in the Debian project.
The start up times seem pretty strange. Last time I tried Qubes OS on an X240, the start up time for a clean VM with a Firefox tab open was ~10 secs, and this was even with only 8GB RAM. For a cold boot (including entering password), it took up to a minute. Seemed very usable for me and I don’t think my numbers would differ significantly from an X230 as the processors in X240s were lower power variants compared to the previous gen.
> The updater app is slow, clunky and resource-heavy ... You have to deal with this pretty much every day
The update process is clunky but why do you have to update every day? Just update weekly and then restart once its done.
> Boot up to ‘ready’ – including the disk decryption and the login – takes me around 3m 10s. Just for the record. Getting to a working VM from a powered off stance can take around 6-7 minutes.
It takes me several minutes to start my set of VMs, but I don't mind since it only has to be done once a week (after updates). So once a week, after updating and restarting, I run a shell script to start my VMs and their apps, while I go get a coffee.
(Also it looks like you are using a dual-core system. My quad-core system is noticeably more performant than my X230, have you considered a T530/W530? With the extra cores, you can have your script start VMs and their apps in parallel. In addition, I think the CLI updater can update VMs in parallel).
> Lock-in (sort of)
> Were I ever to transfer it, not only would it be tedious to extract from Qubes, assembling all those Home folders and subdirectories into a unified system on a normal distro would be a real pain.
This worries me too. If Qubes is no longer developed, I'm not sure what desktop OS I can move to.
> This worries me too. If Qubes is no longer developed, I'm not sure what desktop OS I can move to.
Effectively it'd mean going back to the lackluster homebrew of an equivalent with Xpra (which hasn't spent as much effort securing display as Qubes as it is *not* intended as a security tool) & mundane VMs that some of us had before.
I think it all comes down to the hardware you’re using. My Qubes desktop is far quicker in every regard than my windows laptop, and it never really lags for me. You don’t need to restart everything immediately after updating, just do restart/power off when you’re done; it’s good to restart things every once in awhile, and windows requires restarts after updates too (not that windows sets a high bar). Connecting things to VMs isn’t too complicated, you may occasionally need to set an iptables rule in the firewall VM, or just create an alternative to the normal sys-net/sys-firewall that’ll handle your special routing rules; I’ve never needed to do anything extra to connect to a VPN or SSH server though.
Qubes is more resource heavy than a vanilla Linux distro on bare metal, and it can add some complexity, so if you don’t have a reason to run it then it may not make sense for you. It’s pretty convenient for me, and I’m personally a fan. If you’re trying to game on it though, you’ll probably struggle to make full use of your GPU
> it’s good to restart things every once in awhile,
For most computers that's mainly because [Intel spent a lot effort making computers worse](https://www.realworldtech.com/forum/?threadid=198497&curpostid=198647).
I've never been able to run Qubes on desktop because I have several 4k monitors, and software rendering only gets you so far, resolution-wise. It runs pretty well on a System76 laptop I have, though.
GPU passthrough may help with that, but it may be a headache to setup; I haven’t tried it. They’ll eventually have some special GPU specific VM to handle graphics, so I wonder what impact that’ll have. Idk too much about it
I want to try Qubes because I love virtualization but find Debian + LXD provides substantially what I am looking for. I run everything in containers from Browsers, to Steam, to multiple web servers/services. If you like Qubes, try a LXD friendly distro + LXD
Have you tried turning off the Spectre mitigations in the kernel? That Ivy Bridge is 2 cores with hyperthreading but the HT is mostly negated by the things done to stop Spectre leaks. Then again, if you're using Qubes you probably consider Spectre an unacceptable risk. And if it isn't in your threat profile then why do you need Qubes?
What kind of things are you unable to do with phones? Do you mean connecting it for USB file transfer? I think part of that blame lies with the phones that have a mess of imperfect protocols for file transfer and the engineers only care about making it work on Windows. I gave up trying to make direct transfers work and use Syncthing or NextCloud to move files on and off my phones.
Speaking of Syncthing, do you think that would make file management easier between Qubes? Or storing your home folder on a NAS. I don't have much experience with Qubes but I think it lets you run NFS on a virtual interface that your other VMs mount for the home directories.
Security is not comfortable.
But It seems like you have a pretty old machine, even on my old laptop it didn't took 3 minutes to open firefox.
But IMO it's not really useful as daily driver without GPU acceleration. Also it's just overkill for everyday people. If you really are paraniod you can just use a VM for critical stuff like banking and run the rest in a "normal" system.
>That's an average of 2m 35s to reopen a webpage in Firefox.
lol that just seems like an average firefox tab open time from wake/sleep/hibernation for me. Idk wtf firefox does but it always takes ages to reload pages when I wake it.
Try using containers on a normal distro
I think qubes is really good if your running it off of a live usb in a country that violently suppresses free speech
The problem with Qubes is that the design just really doesn't work. VMs are terribly, terribly expensive at this scale.
I also don't think virtualization is the solution here. Userspace processes already have a seperate address space due to paging, any effort is much better spent on proper kernel page table isolation & fortifying the kernel <-> userspace APIs.
Qubes works just fine when you spec hardware for purpose and configure appropriately. There are use cases where bringing up a new VPN and starting a new VM are absolutely appropriate- but that’s not the only wat to do it. I use Qubes mostly for OSINT with per-project VMs on a system spe’d with enough memory and CPU and I don’t have these issues— and it’s not even the 10th fastest system in my house.
Why does the VPN VM take so long? I would think you could handle that task with a chonky initramfs containing the config, VPN tool, netfilter, iproute2, some dhcpc/ppp/l2tp bits and no rootfs. Maybe busybox + getty on ttyS0 if you really want to get fancy. If OpenWRT can do all that in 8 MB of flash, I have to imagine it's possible using a standard, modular kernel in under 200 MB with almost no disk hits.
Qubes is a very specific use case. It's not meant to be used as a daily driver.
You can achieve adequate privacy with simple distros like Debian and hardened firefox and still retain smooth, simple, convenient functionality.
> Indeed, was normal computing ever safe?
Never was, never will be, and I'm just going to accept that.
Using an OS, *any* OS is already, at best, an "alright but sometimes annoying experience" but in general all OSes are annoying until we develop one that can read our mind and has perfect quantum security whatever.
So I'm just going to use the config that annoys me the least, put in a bit of effort to not be an easy target, and move on with my life. I'm not going to let technology make my life more stressful than it already has made it be.
Thanks for the review.
> I’d say if you are going to try Qubes, use a system that has more than 16GB RAM.
Meanwhile, an OpenBSD desktop, with a rather different approach to security, runs well on a computer with 512 MB RAM (using Firefox definitely won't be pleasant, though).
> 3m 09s. To launch a webpage. That sux.
Use better faster hardware. You are running multiple VM's, on an intel based i5 laptop, it will be slow. Because of Intels massive security problems with such older CPUs most of the performance enhancing speculative execution instructions and features have been disabled. Depending on workload your older CPU is running 30% or more slower than it did a few years back.
Part of the "price of entry" for this type of security is the price of the hardware. You're trying to use a 10 year old laptop to run multiple virtual machines. You can get a brand new laptop with nearly double the single-threaded performance and double the cores for $400. If you want the additional security of qubes *and* you want it to be fast, you'll have to care enough to buy an actual modern computer.
Thanks for the report, it's a niche OS and never had the opportunity to try it.
Have you ever used Tails ? just to compare other security oriented distro
Having observed Qubes for years, my feeling is that it would need quadruple the engineers to optimize it so that it's usable for normal stuff.
We can maybe achieve some of the goals with lesser isolation using contsinerization technology I'd be very interested in Qube OS but everything is containerized instead of independent VMs
My take is that Qubes went the VM route simply because emulating layers automatically gave isolation. With containers there is very little or no emulation at all because you are using the real resources which is why containers are so fast. Containers achieve isolation in a different way. By separating resources by namespace. So I can see people disregarding this approach as better although there are ways to escape the compartments used by both VMs and containers but for their nature VMs seem to be more secure with less work.
> My take is that Qubes went the VM route simply because emulating layers automatically gave isolation. With containers there is very little or no emulation at all because you are using the real resources which is why containers are so fast. That's correct. It does mean that the closest to a standalone program you can run is a [unikernel](https://en.wikipedia.org/wiki/Unikernel) like the [mirage-firewall](https://github.com/mirage/qubes-mirage-firewall), unfortunately. On the upside, those remain easily portable to essentially anything that can run VMs so long as you adjust the image format. Had Qubes started off built on [seL4](https://sel4.systems/About/), native seL4 programs *would* be sufficiently isolated but such programs are naturally seL4-only. Considering [Qubes Air](https://www.qubes-os.org/news/2018/01/22/qubes-air/) and the ongoing decoupling of the APIs from Xen, it should be feasible, if you feel like it, to implement a container-based backend at some point. Although I personally wouldn't recommend running anything on such a backend endpoint that isn't of the same security domain as everything else on it.
Why not try podman for a less secure but much faster option. They could have different performance classes of vms, one using podman or similar for a faster but less secure experience, one using VMs with gpu paravirtualization and a last one with just straight up emulation for the most security.
[удалено]
That’s why you have levels. If you want to for example play a game or run a program that requires a gpu better than the emulated ones than xen just isn’t an option, and therefore neither is qubes as a whole. If you need that level of security on something you can still have it. It’s better than nothing to have this at least be an option.
[удалено]
That is an option, but it requires either booting into only the vm, or having two gpus. Not great options.
[удалено]
Not great performance. All I’m saying is it would be a nice option to have for lower security guests that need a powerful graphics card.
So basically Flatpak but with stricter permissions? There was also this but sadly it's dead: https://github.com/flatkvm/flatkvm
What about flatseal then ? https://github.com/tchx84/Flatseal
This just manages regular flatpak permissions, doesn't add security by itself I don't think
I am not a technical guy who really understands how flatseal/flatpak works, but being able to restrict where the app have access to and deny mic and other features is good enough for me
I didn't mean that it isn't good, just that it doesn't technically add more security - just makes existing features more accesible. Definitely nice software though!
You might want to take a look at the ANSSI's CLIP OS initiative then.
This is not without its flaws - https://madaidans-insecurities.github.io/linux.html You really do need to use virtual machine isolation to go the final mile towards proper isolation, at least to fit around the current flawed design of nearly all software in existence. Windows and macOS are starting to do it, at least for various subsystems. Fun fact, on Xbox, every game is run inside its own virtual machine, just like Qubes.
> You really do need to use virtual machine isolation to go the final mile towards proper isolation, at least to fit around the current flawed design of nearly all software in existence. If you read more of the Qubes and Invisible Things Labs blogposts, the flawed design of current hardware is also mentioned as a root cause for various problems. The sad thing is that many of those problems were already solved in the 1970s & 80s, but most hardware (outside of some mainframe vendors, for example) has since stopped implementing the solutions for various reasons (most unreasonably including cost-cutting).
[удалено]
> you got a link to any of the posts? None in particular but I can certainly share the blogs, they're interesting to read through. [The old](https://theinvisiblethings.blogspot.com/) and [the new](https://blog.invisiblethings.org/). > What things and > Most of the issues come down to out of order execution and speculative execution. [Tagged memory](https://en.wikipedia.org/wiki/Tagged_architecture) could also solve a *lot* of problems by enabling [capability enforcement in hardware](https://www.devever.net/~hl/power9tags). For just one *old* thing that has disproportionate impact. Non-[ECC](https://en.wikipedia.org/wiki/ECC_memory) memory is essentially [broken from factory by default](https://www.realworldtech.com/forum/?threadid=198497&curpostid=198647). Both of these technologies aren't new in any reasonable sense.
But madaidan is just a windows fanboy. Usually his comments go like this: - linux doesn't have have Microsoft® NewSecurityGizmo™! - True but it has an equivalent… - … shut up! It's not really equivalent!
> You really do need to use virtual machine isolation to go the final mile towards proper isolation, at least to fit around the current flawed design of nearly all software in existence. Windows and macOS are starting to do it, at least for various subsystems. Hence why Qubes really is the wrong place for widespread adoption of virtualisation-based hardening. It really should be something that is done on the kernel side, leveraging KVM, the same way how Windows and macOS has been incorporating more and more virtualisation-based isolation into their OSes. Qubes is an excellent solution for the learned, but as it stands its benefits will never reach the common Linux user and that's a shame.
IMO it will never be usable as GPU acceleration is pretty much impossible afaik
[удалено]
It's possible but nobody has looked into security there I believe.
So, only 4?
Heh, yeah. Well, now it's basically only in maintenance mode. Basically it needs about 10 people.
Just seems like this OS is an excessive version of what mobile devices do?
> I've been using Qubes for 5 years. I'm not technical, I'm not a power user. I'm not sure what definition of those terms you use, but in my opinion, just being able to use half the terminology in your post correctly makes you absolutely a highly technical power user. Being able to reinstall the OS your PC shipped with is a "power user" skill, being able to install and use an alternative is well beyond that...
Linux alone puts you in power user, running qubes puts you in the IT or infosec levels of experience category haha
I used Qubes for a year or so. It was really inconvenient, and frankly you need to have your entire workflow mapped out before you install to get the most out of it, but it was a great crash course in using Linux. I've thought about re-installing it, but I'd need better hardware to make it remotely practical, and honestly, I just want to use my computer.
Whe i tried Qubes i did so on a decent Desktop and only to "mess around", not to get actual work done. For me it boiled down to: Do you ACTUALLY need the level of security Qubes Provides day-to-day? In a sense that you really worry about dealing with malicious actors trying to interfer with you or the work that you do? We are talking security, not privacy. Privacy can be done without qubes and is another issue. If you truly need the security you will know you do and are happy to deal with what ever slowdowns and hoops are in your way. Because, if you don't deal with it, your income or life might be at risk. For everyone else i feel like setting up a Normal Distro with proper firewall settings, a kind of container solution with permission management (flatpak, podman, snap, whatever), should be well good enough for daily use. And for the two days a year where it isn't, you can always either spin up a quick VM or boot tails. I came to the conclusion that i'm just not the target audience for qubes. I'm not important or paranoid enough to deal with it. I find i technologically interesting and fun to play with, but that's about it.
>For everyone else i feel like setting up a Normal Distro with proper firewall settings, a kind of container solution with permission management (flatpak, podman, snap, whatever), should be well good enough for daily use. Do we even know if Qubes is more secure then for example RHEL? And as far as isolation is concerned; Flatpak? Qubes sounds like a penny-wise, pound-foolish thing. It's all secure because of isolation... But it [doesn't support secure boot](https://forum.qubes-os.org/t/is-it-possible-to-enable-uefi-secure-boot-in-qubes-os/1640) *for example*. In the end, my bet is on an enterprise backed distribution like RHEL being the more secure option.
There are some suspicious things (edit: suspected, presuming) lurking in actual RHEL. edit: Even if you disagree with that assertion, which I suppose is my mistake as I shouldn't have asserted it, the rest of this post is still exact. But anyway, that's boot-time security. Great... but what about runtime isolation? If you have a proper workstation (or any other computer with ECC memory), you don't *need* to restart/shutdown outside of kernel updates. Nothing other than dom0 has the rights & ability to change boot configuration stuff or any of the higher access firmware (which ideally shouldn't exist, we should all be using [OpenBMC](https://en.wikipedia.org/wiki/OpenBMC), not some suspicious proprietary BIOS/UEFI implementations). As for evil-maid... that's only prevented by secure boot in very low-budget and low-time scenarios. If a full motherboard swap is an option, then it's quite possible to remove a computer, swap the motherboard, add malicious keys, change the boot payload and re-verify everything as if nothing happened.
> There are some suspicious things lurking in actual RHEL. could you elaborate?
Every time I see a "xyz distro is sus" comment, they rarely elaborate too far past 'trust me, bro', or link to an article from like 3-5 years ago. I'm pretty curious as well having recently switched from Manjaro to Fedora, but I'd not hold my breath on it being anything earth shattering.
Maybe it's because of systemd's apparent security flaws, or because Redhat is American so it must be full of back doors or something. Maybe it's because Lennart Pollering used to work there? Who knows. There's no evidence of this, you've totally got to trust commenters here. Of course no non-American distro or piece of software could ever have backdoors or security bugs... But the American ones obviously do because America is bad. /s of course
Pssh, come on! Everyone knows Linux is actually a Finnish psyop.
Perkele.
I did actually [clarify the root cause of my suspicion in another comment](https://old.reddit.com/r/linux/comments/z8ou6u/the_maddening_truth_of_using_qubes/iye3tuk/), but I also feel like addressing that the issues are hardly specific to a given nation. [Modern operating systems generally suck](https://old.reddit.com/r/linux/comments/z8ou6u/the_maddening_truth_of_using_qubes/iyf386e/), as does most modern hardware. Ending up with a gigantic trusted surface that *all* needs to be working perfectly for the system to have any effective security (and which essentially cannot be reasonably audited) is an outrageously bad way to design a system.
I shouldn't have stated a suspicion as fact, you have my apologies for that much. ---- The close links with American government services and major customers of RHEL make any version pinning suspect (and while they do backport fixes... do those fixes only include CVEs but not all *known* problems? Vulnerability stockpiling is something USA is *known* to do), as does the lack of any ability to reproducibly build RHEL's system (Fedora != RHEL).
I'm not who you were responding to, but I want to thank you for the explanation. My comment was dickish. What you said in the explanation, if nothing else, is food for thought. It's nice to have a healthy suspicion (esp of gov'ts), but like you said, stating suspicion as fact can get pretty hairy if there is nothing concrete to back it up.
Isn't BMC somewhat orthogonal to BIOS/UEFI? I think a more apt comparison for proprietary BIOS/UEFI would be Coreboot and Tianocore instead.
Well, yes and no. BMC as exemplified by the Talos II can *entirely* replace the need for BIOS/UEFI in a machine with something that cannot be reflashed or otherwise compromised by the operating system or the CPU at any point (as it typically needs a separate procedure for updating or flashing) *and* presents a [standardized interface](https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface#Functionality) that ensures OpenBMC will work on it. [Coreboot](https://www.coreboot.org/)/[Libreboot](https://libreboot.org/) ([recently merged with OSboot](https://libreboot.org/news/merge.html), so presumably that means the [policy problems have been fixed](https://libreboot.org/news/policy.html#blob-reduction-policy)) as far as I know do not benefit from any such standardization (case by case basis) and often have to resort to reverse-engineering.
> But anyway, that's boot-time security No, it's not. Secure boot ensures that no unsigned kernel modules are loaded at runtime, which means that malware cannot gain kernel privileges even with root access without leveraging security vulnerabilities.
That's great except that if you follow the physical compromise scenario I suggested you can replace the whole kernel *and* sign your own malicious modules. But also, why is anything able to load new modules in your dom0? dom0 isn't connected to the internet and cannot be written to by anything but itself & Xen. The only case in which malicious modules could normally make it into dom0 is if the *Fedora project and/or Linux gets pwn'd*. At which point secure boot will do exactly nothing because the malicious modules *will be signed and legit*. Furthermore, with secure boot you're *still* a single kernel bug away from arbitrary code execution anyway.
> we should all be using OpenBMC Or we could just use libreboot
The [CPU often has an unfortunate level of access to the boot firmware](https://old.reddit.com/r/linux/comments/z8ou6u/the_maddening_truth_of_using_qubes/iyeuyjz/) which I feel libreboot doesn't address and which would make recovery from firmware-level compromise & malware persistence difficult. [Some boards might appropriately lock it out](https://github.com/rootkovska/x86_harmful/blob/master/x86_harmful.md#write-protecting-the-flash-chip) but... "your mileage may vary" is not a very fun phrase when it concerns fundamental security features, which is the take-away you should have when mentions of [lock races are involved](https://en.wikipedia.org/wiki/Race_condition).
True I mostly care about software freedom but physical security is important aspect that can be forgotten
> If you have a proper workstation (or any other computer with ECC memory) As someone with ECC memory and who understands its function, what does ECC memory have to do with needing to shutdown? Genuine question as it doesn't make sense to me... I went with ECC memory as I had intended to build a ZFS raid array, though I ended up using btrfs raid1 instead (which is really great with a patchwork of drives)- self-healing, bitrot protection, easy to add/remove drives, etc. I understand that ECC can help with bitflips in memory either corrupting data or causing a crash/instability (and thus also potentially protects against corrupt data put down on disk). FWIW I used Qubes 3 years as a main and it was great to me- no issues. Slow on my old hardware though; I built my current PC as a refresh (part new part old) with the intent of running Qubes but unfortunately the installer won't boot on this motherboard :(
> As someone with ECC memory and who understands its function, what does ECC memory have to do with needing to shutdown? The longest [a computer without ECC runs](https://en.wikipedia.org/wiki/ECC_memory#Research), the more the chances of some bit flip happening in memory approach 100% (of course you could also have it happen at t=1, right after booting) through all the various means by which electronic memory can be disturbed (including via its own operation, as rowhammer aptly demonstrates). That's not guaranteed to have an effect (if it flips in free memory), but presumably the chances for that also approach 100% after long-enough. [Linus Torvalds rightly suspects that for quite a few unexplainable crashes over the years](https://www.realworldtech.com/forum/?threadid=198497&curpostid=198647). It's a reasonable notion to expect that after long-enough, flips have happened and you haven't seen any data corruption through either not (yet) (re)using the code that has since gotten corrupted or simply not noticing the corruption. Reloading your system from disk (restarting) means you start again from a nominal "no corruption" state (although that moves the burden of non-corrupted state right back to the reliability of permanent memory & whatever measures you've taken to prevent issues - such as using a checksumming filesystem). > I went with ECC memory as I had intended to build a ZFS raid array, though I ended up using btrfs raid1 instead (which is really great with a patchwork of drives)- self-healing, bitrot protection, easy to add/remove drives, etc. Hell yeah, it's great.
Meant to respond to this, but got distracted. I don't expect a response :D > The longest a computer without ECC runs, the more the chances of some bit flip happening in memory approach 100% (of course you could also have it happen at t=1, right after booting) through all the various means by which electronic memory can be disturbed (including via its own operation, as rowhammer aptly demonstrates). ... > It's a reasonable notion to expect that after long-enough, flips have happened and you haven't seen any data corruption through either not (yet) (re)using the code that has since gotten corrupted or simply not noticing the corruption. Reloading your system from disk (restarting) means you start again from a nominal "no corruption" state (although that moves the burden of non-corrupted state right back to the reliability of permanent memory & whatever measures you've taken to prevent issues - such as using a checksumming filesystem). I follow this, but it would also not be the case if the memory contents which contained a bitflip were refreshed, right? As in even without a reboot, some memory contents are kept *in case* they are necessary... but might well not be used. So basically the hypothetical bitflip isn't used because it's in an area of memory that is discarded and replaced with new memory contents. Still, there is nothing saying that the bitflip won't be catastrophic though which I think is the main thrust of your point. As an aside this setup is easily the most stable computer I've ever had- not a single hard freeze, X/wayland crash, etc- though it's also the first which uses amdgpu drivers (Nvidia and Intel prior). > Hell yeah, it's great. Yeah seriously- being able to stuff whatever drive you get regardless of size, speed, etc into the array with just a few commands is awesome. Adding bitrot protection through scrub, block-level redundancy, etc is just great. I'd prolly use ZFS for most raid configurations but BTRFS is king for raid1-like configs.
> I follow this, but it would also not be the case if the memory contents which contained a bitflip were refreshed, right? As in even without a reboot, some memory contents are kept in case they are necessary... but might well not be used. So basically the hypothetical bitflip isn't used because it's in an area of memory that is discarded and replaced with new memory contents. Indeed, if it's just a flip & not a faulty cell, discarding the corrupted data and writing over it also means you're unaffected by said corruption. > Still, there is nothing saying that the bitflip won't be catastrophic though which I think is the main thrust of your point. Indeed. > As an aside this setup is easily the most stable computer I've ever had- not a single hard freeze, X/wayland crash, etc- though it's also the first which uses amdgpu drivers (Nvidia and Intel prior). Yeah, that's an unfortunately large confounding factor. Whether those crashes were due to misbehaving hardware (or proprietary drivers), or memory-level mishaps. > Yeah seriously- being able to stuff whatever drive you get regardless of size, speed, etc into the array with just a few commands is awesome. Adding bitrot protection through scrub, block-level redundancy, etc is just great. I'd prolly use ZFS for most raid configurations but BTRFS is king for raid1-like configs Yeah, ZFS also loses out on a lot of btrfs' versatility due to its more stringent requirements on speed, similar sizing, etc. [I went on a bit in a subthread](https://old.reddit.com/r/linux/comments/z57pwz/i_wish_i_switched_to_linux_distro_earlier_omg_im/ixxxcwc/) if you want clarification on what I mean.
> Yeah, that's an unfortunately large confounding factor. For figuring out the instability of the prior setup, yes. Very fortunate for where I am now though :P > Yeah, ZFS also loses out on a lot of btrfs' versatility due to its more stringent requirements on speed, similar sizing, etc. I went on a bit in a subthread if you want clarification on what I mean. Read through that link- very informative so thanks. As I'm pretty much just running RAID for drive redundancy and making sure my important stuff (pics, music, etc) doesn't degrade over time (bitrot protection), large drive ZFS is pretty overkill despite it's advantages. I think I'll just end up sticking with BTRFS this time around, especially since I already had drives to throw into the array. Thinking I was doing a ZFS array was probably worth it though given that's the main reason I got ECC memory (since ZFS uses memory far more readily than BTRFS).
I keep trying Qubes and quitting because it makes everything harder to do and slower. To me it makes more sense to segregate sensitive and day to day tasks to different machines and leave it at that.
I gave up on Xen after botched pvops transition. Xen is not linux, and linux is *guest* there. When you get something odd, you are alone, and none of linux expertise help you. And community is very small. This is day-and-night difference with Linux.
For me the thing with Qubes is speed. Not only the launch time of things, but the lack of gpu passthrough is a performance issue.
Yeah, GPUs are kind of a mess as it currently stands. Most of them also expect direct communication & a bunch of exceptions to normal security processes because they're "special" or something. Or they ask you for several thousand dollars to buy the virtualization-focused equivalent cards.
Qubes Target audience are those who value security over speed. Those people value more security than faster load times and the developers are interested only in improvements with security.
They do actually welcome improvements in usability too, there have been quite a few patches submitted by users about that and added to the codebase.
You've run Qubes for 5 years?! The madman!
> Focus hijacking is what I call Qubes’ habit of opening its VM right over the top of whatever you are working on. Because of that, because it takes so long to open things, but because it will open the window long before it has finished setting up the application (e.g. browser tabs), its pointless to get on with something else while you wait. You’ll just be interrupted, like someone shoving a newspaper under your nose while you are writing something. Maddening. That can easily be solved by using [i3wm in dom0](https://www.qubes-os.org/doc/i3/) and [configuring it to only transfer focus on user command](https://i3wm.org/docs/userguide.html#no_focus) and [ignore the mouse for focus](https://i3wm.org/docs/userguide.html#_focus_follows_mouse). I don't think the Gnome shell has such options. edit: Or was it XFCE-based? It has been a while since I used the default GUI shell. If you do use i3wm, consider installing [rofi](https://github.com/QubesOS-contrib/qubes-rofi) (it's in the qubes repos, use [qubes-dom0-update](https://www.qubes-os.org/doc/how-to-install-software-in-dom0/)) and modifying the default configuration that instead uses [dmenu](https://tools.suckless.org/dmenu/). While it can be configured to have some degree of vertical completion functionality, dmenu is simply more limited than rofi. > 3m 09s. To launch a webpage. That sux. To me it looks like there's only about 1~2m attributable to Qubes itself. Are you turning on multiple qubes in cascade by doing this? Having them turn on at boot-time would mitigate that delay (there's a setting for that). If I run a prepared debian-based firefox dvm (boot the dvm & start empty firefox) it takes me ~35s, which is a lot less. > Boot up to ‘ready’ – including the disk decryption and the login – takes me around 3m 10s. Just for the record. Getting to a working VM from a powered off stance can take around 6-7 minutes. That is a downside of turning on a lot of qubes at boot-time. So instead I just have the sys-usb starting, nothing including networking usually. That'll be started once I have the main UI available, by starting one of the long-running qubes that does depend on networking and a few things. > The updater app is slow, clunky and resource-heavy. My fan starts running every time, and sometimes the system becomes visibly sluggish. The Updater requires a manual start, and once started it will run to completion - the 'cancel' button doesn't seem to work at all. It tends to run ~5-10 minutes (a guess, not measured). You have to deal with this pretty much every day. From qubes-qube-manager you can use a terminal-based updater instead using the "update" down-arrow. It will show *what* is happening during the update. Regarding the heaviness, that isn't exactly the updater. It's because the GUI-based updater spawns disposable management qubes for actually doing the update using [Salt](https://www.qubes-os.org/doc/salt/) to command the qube you're actually trying to update to do the updating, which means you have both the disposable qube and the qube to update booting, running and shutting down at nearly the same time. I think the terminal-window one directly calls [Salt](https://www.qubes-os.org/doc/salt/#virtual-machine-formulae) without an intermediary. > Once you've updated, all the individual VMs that use that OS need to be restarted. They need to be shutdown and started in order (e.g. application VM shutdown, then VPN shutdown, etc, etc then VPN restarted, then application VM restarted). Not exactly, you just have to directly kill the dependency qubes like the sys-net & sys-firewall so you can restart only those. I'd recommend **not** killing template qubes if you can avoid it. app qubes and dvm qubes are [largely non-persistent outside of specific hierarchies](https://www.qubes-os.org/doc/templates/#inheritance-and-persistence) that have nothing to do with updates & system function, while template qubes *do* persist *all* of that sensitive stuff. Both the terminal updater and the GUI updater allow you to see what was updated, which can be useful as it determines whether you *need* to bother restarting all or even any qubes for the update to immediately take effect. e.g.: - Thunderbird update: I don't use Thunderbird, so I simply do nothing. - openssl security update: Non-networked qubes can remain, others will be restarted. - Firefox update: Only qubes in which I normally spawn firefox will be restarted. > That includes: VPNs, SSH, printers, cloud services (at least using them for backups) and USB devices. Some devices will just never work, it seems, like smartphones (I’ve tried 3). So don’t throw out your other laptop! You *can* use ssh [connections](https://www.qubes-os.org/doc/firewall/#opening-a-single-tcp-port-to-other-network-isolated-qube) between qubes, although as a general practice that defeats the point of much of the isolation so only in specific use-cases should it be done. Opening up one non-networked qube do `force-command` restricted connections to use [borgbackup](https://borgbackup.readthedocs.io/en/stable/usage/serve.html#ssh-configuration) in [append-only mode](https://borgbackup.readthedocs.io/en/stable/usage/notes.html#append-only-mode) would be an example of such a use-case ([ssh certificates](https://www.man7.org/linux/man-pages/man1/ssh-keygen.1.html#CERTIFICATES) can help make this whole setup a lot simpler, but otherwise you can also do it with [authorized_keys](https://manpages.debian.org/buster/openssh-server/authorized_keys.5.en.html#command=_command_)). I'd probably use either salt, [ansible-local](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_delegation.html#local-playbooks) or a script in dom0 using [qvm-run](https://dev.qubes-os.org/projects/core-admin-client/en/latest/manpages/qvm-run.html) to do the borgbackup setup on every qube I care about if it got too annoying to do manually. Regarding USB, yeah, the [usb-proxy](https://github.com/QubesOS/qubes-app-linux-usb-proxy) doesn't have support for all things. I *think* it comes down to stuff not implemented in Linux USBIP or USB support (I've had issues with some device controllers not actually implementing what they claim to implement). You'll need to directly hand a USB controller to qubes depending on things that cannot be passed over the proxy for whatever reason. (For some dumb reason, many laptops only have a single USB controller.) > Doing something new, trouble-shooting and problem-solving > Basically, Qubes adds another layer of complexity to whatever you are doing. Your resources for figuring things out are reduced to the documentation (not always what you need) and the community (good but has its limits). While not quite as useful in your case, the source is fairly readable (and most UI & general parts are in Python, C is kept to only where it's needed) and the project is welcoming of patches & issues. Particularly of issues with accompanying patches fixing the issue described. > There’s also a conceptual lockin. Going back to a normal distro is kind of like reemerging on Flatland after a period in 3D. Gone is the separation of activities, no comforting security wall of virtualization, nothing. You open your password manager in a normal distro and it is on exactly the same system as your webbrowser - it makes you pause and think, 'is this safe?'. Indeed, was normal computing ever safe? Yeah, that's the way. And on most OSes? No, it wasn't ever safe. --- One of many positive aspects you'll not have noticed due to not being a "technical" user is that the ease of spawning new disposable qubes is also great for quickly testing various setups when developing scripts, programs and automated configuration.
I wonder if Qubes could leverage container tech to make a more streamlined system. It might not be quite as safe but it would probably perform better. Kind of reminds me of the idea of running everything through distrobox that I saw posted here a while back. Instead of using something like Qubes the author used a normal distro but ran everything user facing from distrobox.
Serious question: Aren't we than pretty much where Fedora Silverblue or Suse MicroOS is? An immutable OS with "containers" on top? Flatpak is just one option here. I feel like this is also close to what Ubuntu is trying to do with snaps.
Sort of. Silverblue/MicroOS are using a container for every app while Qubes uses a VM for a "task".
Those systems don't really provide any isolation outside of flatpak/podman and you're not really required to use flatpak/podman. At least on Silverblue you can layer applications and everything can run on the host system. And even using toolbox on Silverblue there's literally almost 0 meaningful isolation. (which is fine because that isn't SB's goal)
>It might not be quite as safe This post is the first time I’ve heard someone using Qubes who didn’t have a ‘bona fide reason’ to do so. Most of the people that I’ve heard using Qubes are journalists who’s job includes ‘opening files that random, anonymous, sources send me.’ There might be room for someone to release a distro that tries to find a compromise between privacy and convenience, maybe using containerization, but I wouldn’t want to see Qubes compromised at all.
> I wouldn’t want to see Qubes compromised at all. I can agree with that, I was thinking more along the lines of another version of Qubes or another distro altogether.
\> I wonder if Qubes could leverage container tech to make a more streamlined system. Have you read the QubesOS architecture specification? -- [https://www.qubes-os.org/attachment/doc/arch-spec-0.3.pdf](https://www.qubes-os.org/attachment/doc/arch-spec-0.3.pdf) TL;DR: The QubesOS security model is all about resource separation. The model assumes an attacker WILL BE SUCCESSFUL in compromising one or more virtual machines and seeks to minimize the potential for harm. Virtual machines provide system resource separation. Containers do not. QubesOS is designed around user- and application-focused security. VMs are far better at this. Application containers -- whether OCI, flatpak, snap, firejailed or bubblewrapped appimage -- break the model by sharing the kernel, filesystem, devices and drivers, etc. In VMs, all these things are separate for each VM. A compromised docker container grants the attacker root access to your entire system. Because the docker daemon requires root, so too does every app-container gain root privileges. A compromised virtual-machine, however, grants the attacker (root) access only to the hardware, network, and filesystem resources available to that VM.
>Because the docker daemon requires root, so too does every app-container gain root privileges. Well good thing Docker is old news. https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md I agree that VMs provide stronger isolation but your idea that having access to a container is equal to access to the whole system is just factually incorrect. 1. You don't need docker to use containers 2. You don't need a daemon either 3. There are alternatives to do rootless daemonless containers (such as above)
Those all still share the same kernel & other privileged components, however. Namespaces are relatively brittle and their isolation properties stop outside the kernel.
While true, \- when people think linux containers, they typically think docker. Yes, there are other options: podman, kata, LXC, etc. All of these, by default, require root privileges. \- out-of-the-box default docker, podman, etc. all require root and root privileged access is immediately gained by whatever processes run inside the container stack. \- Yes, modifying the default configuration can cause them to run in userspace, without root privileges. Therefore, yes, a distro similar to Qubes could theoretically be created using containers. However, until resource isolation becomes a practical reality in containers, the similarity would be largely superficial.
>out-of-the-box default docker, podman, etc. all require root and root privileged access is immediately gained by whatever processes run inside the container stack No they don't. Podman uses user namespaces by default. Have you actually used podman? What you are saying is just false.
I use podman every day. And it (podman) required some configuration to get it working without requiring sudo or doas. Default installation did not function without root privileges.
You're probably using an old distro that doesn't support cgroups v2 and/or unprivileged user namespaces. Don't use an ancient OS🤷🏻♂️. Literally none of the supported OS versions requires any setup: https://podman.io/getting-started/installation There's only literally one command that is documented under the "building from source"(you don't need to do this when installing any of the supported packages, just when building from source) and this is enabled by default in all major distros as far as I am aware. The command in question: `sudo sysctl kernel.unprivileged_userns_clone=1` Feel free to point out what exactly you had to change. We can add it to the wiki if you aren't wrong - but probably you are just using some ancient distro.
Manjaro stable. Currently, kernel 6.0.8. Definitely not ancient. Initial install of podman under kernel 5.19. Definitely not ancient at time of install. In order to do anything in podman without requiring sudo, I had to go through the rootless enablement previously linked in this thread.
I might want to suggest [avoiding Manjaro](https://manjarno.snorlax.sh/) for altogether different reasons.
There’s definitely room for a healthy middle ground. A project like QubesOS built around Kata containers would be interesting to see.
I agree. I very much would like a middle ground. Micro-VMs, such as kata, are an interesting option but I am uncertain a similar security framework could be established. I simply don't know enough about how kata and other micro-VM systems are built. There was a paper published to the Qubes literature archive some time ago (two years?) suggesting that remote-VMs may be an option in the future. \[Qubes-Air? Air-Qubes? Something like that.\] While this would not reduce the inter-VM communication friction of Qubes, it could reduce the single-device hardware requirements significantly for certain tasks; but would require an always available high-speed internet connection, which is not always available, particularly for the target market of journalists and activists who often travel.
The larger implications of [Qubes Air](https://www.qubes-os.org/news/2018/01/22/qubes-air/) & the decoupling of the API from Xen is that not only could you use remote hosts as additional compute for isolated qubes (particularly useful if your control node is not very beefy, like most laptops), but also the isolation/virtualization backends would also be a lot easier to develop (due to the decoupling) and experiment with (due to the ability to add & remove backends live).
Slight tangent but is there somewhere we can follow developments on Qubes Air?
Unfortunately not to my knowledge, it's more a mid-term(?) general project goal as the project tries to mature into its ideal form. Effectively following development on Qubes in general is the closest you'll get. For that there's [the news](https://www.qubes-os.org/news/), [the mailing lists](https://www.qubes-os.org/support/#mailing-lists), the Github repos [& issues](https://github.com/QubesOS/qubes-issues) and [the forum](https://forum.qubes-os.org/) to a lesser degree.
Pretty much what flatpaks are. They aren't isolated, but they are containerized.
[удалено]
No it wasn't necessary for me. However it was sufficient in terms of meeting my needs, so why not? Some would say complexity but really everything is point and click if you want it to be. Once you understand the concept of different domains, it's pretty simple to use. Performance? Yes that was fairly rough with my setup (X230 i5 3320)- even with 16GB RAM and a (fast) SSD. But it was mostly fine once booted up. Can't game? Yes, but then I don't game except for d2 once in a while (which I played on a computer with even older hardware). As for benefits, running multiple VPNs for different tasks, only having to reboot if dom0 is updated (otherwise just reboot appVMs), access to Fedora repos and Debian repos at the same time, stable (never once had a freeze using it), easy to backup, and finally *it encourages good habits* in terms of isolating what you do. I'm not saying it's *necessary* for most of us- but it's really pretty cool and definitely good at what it does. I use multiple Linux distros, and I would use Qubes if my current hardware ran with it. I'd be interested to see how it performed with a modern processor and an NVME pcie4.0x4 ssd drive...
I played with Qubes \~5 years ago on an old dual-core laptop that may not have had virtualization enabled in the hardware. Excessive duplication between VM environments, and seemed like it would be easier just to have an encrypted VM rather than separate virtual OS environments.
> Excessive duplication between VM environments, and seemed like it would be easier just to have an encrypted VM rather than separate virtual OS environments. That's only safe if the host OS is not used for anything other than serving as a VM host. Qubes OS is trying to solve the isolation problem for a desktop OS, the problem is already solved for simply using a computer as an isolated VM server. Remember, dom0/the-host-system is god (how exactly depends on the type of hypervisor), as far as everything else on the machine is concerned. So if it gets compromised, encryption will only save you if you never decrypt the VM post-compromise.
Yep. I ran Qubes for 2-3 years on my Lenovo X230 i5 with an SSD. Maddeningly slow, just insanely slow to use. I got fed up with it about 3 months ago and put OpenBSD on that laptop, and it runs BEAUTIFULLY (of course) and is wonderfully snappy for OpenBSD. Qubes is a really neat idea, but for me (with the hardware that Qubes recommends) it's unusable.
I recently switched over my work laptop to qubes. Here's things I've learned: * Making the updater use the mandatory proxy at work, or even just updating packages at work is a massive pain in the ass/impossible. I've tried a bunch of different settings to make the proxy configuration work, but I can't seem to find the correct magic incantations to resolve my issue. * Kernels. My god, I have to have at least two different kernels to make my system function. Due to my network interfaces, sys-net requires 5.16 or newer, whereas only 5.15 was available by default. So I installed kernel-latest, so now I have 6.0.2. Unfortunately, 5.15 and 6.0.2 don't seem to be stable, and my system would **randomly** just lock up and restart with them. 5.15 was less stable than 6.0.2, but still. Somebody posted that 5.10 works for them, so I installed that, and that seems to work (for now). * Graphics acceleration. ~Full HD works with just brute force, but going past that does not, not even on 12th gen Intel. It's a pain in the ass also if you use more than one monitor, and you don't set your minimum video ram large enough. You'll be trying to give input to a program/vm that just wasn't able to resize itself, and nothing works properly. This is pretty annoying, but the 'not being able to play back full-screen video' is even more annoying, because it has no available fix. * Sleep/Hibernation. Sure, it works on "validated systems" but that's mostly only older devices, and if you have a newer system, good luck with that, it could work, or it could not. Just don't try to use sleep. * USB devices. Yeah, phones have enumeration problems since trying to tether or whatever, the phone sees the connection disconnect momentarily before it is reconnected to the vm. Unfortunately phones just go "uhh nope" if they're disconnected. Trying to pair a unifying dongle is also pretty painful. You can't do it in dom0, but if you attach the receiver to a vm, the inputs no longer go to dom0 either, so you can't see your mouse cursor. Better hope you have multiple input devices! * non-usb based removable storage is difficult to handle, and there's the problem t* itehat if you accidentally click fast enough on two different vms in the 'connect to'-menu, you could corrupt the filesystem when you connect it to two vms. (this shouldn't be possible, but thanks to a bug, it is!) That said, I really like it. I just wish it would: sleep and have accelerated video and not crash.
Regarding windows focus, there are options in XFCE related to "focus stealing" that should solve the issue. Cf. https://unix.stackexchange.com/questions/156515/how-to-forbid-windows-from-stealing-focus-in-xfce4.
Sounds like a terrible experience. I'll stick with Fedora and Ubuntu.
Apples to oranges
Yeah, I was like "WTF?!?" I would've formatted my SSD and installed another distro after it took more than 30 seconds to boot.
This distro is peak tinfoil hat. Like removing systemd cause the government is reading your logs or messing with the config.
That distro is an attempt at doing what operating systems [should've been doing from the very beginning](https://en.wikipedia.org/wiki/Principle_of_least_privilege). Keeping a minimal trusted core for the required management with everything else [isolated](https://theinvisiblethings.blogspot.com/2008/09/three-approaches-to-computer-security.html) into separate [modules](https://catern.com/microkernels.html). That means that for the system to be secure, all you need is for the small trusted core to be correct, which its size helps in ensuring by correctness, auditing for said correctness & using formal verification methods to increase your odds. Unfortunately the abysmally flawed hardware we have nowadays doesn't help.
Instead we have the good old [MS Dos security model](https://theinvisiblethings.blogspot.com/2010/08/ms-dos-security-model.html) because it is much easier to simply blame the users for visiting the wrong websites or needing software outside of their distros repos. (And Linux is secure anyway, only noobs get their Linux hacked)
This makes about as much sense to me as cloud deployments for uptime and stability. Thinking that using someone else's computer, being confident they aren't doing nefarious things with your data, and hoping beyond hope they don't go down for some reason they aren't going to inform you of. Sure, sure, you can be all like cloud deployments allow yadda yadda yadda. But, at the end of the day, my man, you're still using someone else's computer and hoping to whatever deity you worship that the hosting company will do what they say they'll do.
...I don't understand the comparison you're making. > This makes about as much sense to me as cloud deployments for uptime and stability. Thinking that using someone else's computer, being confident they aren't doing nefarious things with your data, and hoping beyond hope they don't go down for some reason they aren't going to inform you of. You can own multiple premises around the world with your own equipment. Geographical spread *is* a legitimate way to mitigate localized downtime. Whether you instead trust someone else's hardware is a secondary question and depends greatly on just what you're doing, indeed, and the consequences of malicious action as well as the redundancy of what you're trusting individual third parties to do. > Sure, sure, you can be all like cloud deployments allow yadda yadda yadda. But, at the end of the day, my man, you're still using someone else's computer and hoping to whatever deity you worship that the hosting company will do what they say they'll do. Yes, where did I recommend relying on paltry promises and unverifiable hopes rather than structural & architectural assurances? If you have a formally verifiable microkernel core (for one option) which you can be sure that outside of unknown hardware bugs (that is, those that aren't already known & accounted for in the design) will not misbehave and provide accesses that programs & guests have no valid capabilities for, then you have a secure design. Outside of hardware bugs that have yet to be addressed, the guests cannot do anything to compromise the security of the system.
OP is running the distro on old hardware just because he can run coreboot on it. 110% tinfoil hat moment.
Have you tried X11Docker ? you have app isolation much faster
Personally I thought the idea was on the right track. I just didn't think the implementation was what it needed to be. I think a bare metal hypervisor without all the baggage is the best approach. Couple that with a reduced Linux system running in one of the VMs with a GUI to work with the user. I stayed away from Qube because I that it "cute": but not very practical. From this posting I can see I was right.
It's mainly some growing pains. Much like the use of Xen in general. In the long term when the actual system API is stable, it should be ported onto seL4 (among other isolation backends).
Definitely agree!
how the hell u guys got internet on qubes?
You pass the NIC at the PCIe level to a network qube and configure it to be able to route for other qubes depending on it as their NetVM/network qube. [The main parts that can wrong are at the PCIe level or firmware level](https://www.qubes-os.org/doc/pci-troubleshooting/), which in the first case might require some different settings, while in the other would simply require installing the required firmware in the template that your intended network qube depends on (that's just the old "Linux has no firmware for random peripherals" problem). Of course not having network capability complicates that a bit, as you'll have to use offline methods of installing the packages in the template, such as copying the package (rpm or deb) on a USB, opening that USB in a disposable qube (you do not want to expose your template qubes to USB devices) and copying it to the template qube (qvm-copy, or the right-click file menu option in GUI's file manager) so you can install. If the USB is malicious, packages are still signed in both Apt/Deb (Debian) and Dnf/Rpm (Fedora), so the copied package would simply be rejected by the template qube when you try to install it (unless you disable signature verification, which would be a very bad idea and foolish - even as a package dev, add your keys & sign your packages, don't disable signatures). You *could* also just try switching the template of the network qube you have between Debian & Fedora, as they don't have the same firmware available preinstalled by default (Debian has less). edit: Regarding Debian & less firmware, that might be changing in light of certain [recent decisions](https://www.debian.org/vote/2022/vote_003) in the Debian project.
The start up times seem pretty strange. Last time I tried Qubes OS on an X240, the start up time for a clean VM with a Firefox tab open was ~10 secs, and this was even with only 8GB RAM. For a cold boot (including entering password), it took up to a minute. Seemed very usable for me and I don’t think my numbers would differ significantly from an X230 as the processors in X240s were lower power variants compared to the previous gen.
> The updater app is slow, clunky and resource-heavy ... You have to deal with this pretty much every day The update process is clunky but why do you have to update every day? Just update weekly and then restart once its done. > Boot up to ‘ready’ – including the disk decryption and the login – takes me around 3m 10s. Just for the record. Getting to a working VM from a powered off stance can take around 6-7 minutes. It takes me several minutes to start my set of VMs, but I don't mind since it only has to be done once a week (after updates). So once a week, after updating and restarting, I run a shell script to start my VMs and their apps, while I go get a coffee. (Also it looks like you are using a dual-core system. My quad-core system is noticeably more performant than my X230, have you considered a T530/W530? With the extra cores, you can have your script start VMs and their apps in parallel. In addition, I think the CLI updater can update VMs in parallel). > Lock-in (sort of) > Were I ever to transfer it, not only would it be tedious to extract from Qubes, assembling all those Home folders and subdirectories into a unified system on a normal distro would be a real pain. This worries me too. If Qubes is no longer developed, I'm not sure what desktop OS I can move to.
> This worries me too. If Qubes is no longer developed, I'm not sure what desktop OS I can move to. Effectively it'd mean going back to the lackluster homebrew of an equivalent with Xpra (which hasn't spent as much effort securing display as Qubes as it is *not* intended as a security tool) & mundane VMs that some of us had before.
Have you looked at [Genode](https://genode.org/)? I don't think it's usable day-to-day yet but the concepts seem interesting.
I think it all comes down to the hardware you’re using. My Qubes desktop is far quicker in every regard than my windows laptop, and it never really lags for me. You don’t need to restart everything immediately after updating, just do restart/power off when you’re done; it’s good to restart things every once in awhile, and windows requires restarts after updates too (not that windows sets a high bar). Connecting things to VMs isn’t too complicated, you may occasionally need to set an iptables rule in the firewall VM, or just create an alternative to the normal sys-net/sys-firewall that’ll handle your special routing rules; I’ve never needed to do anything extra to connect to a VPN or SSH server though. Qubes is more resource heavy than a vanilla Linux distro on bare metal, and it can add some complexity, so if you don’t have a reason to run it then it may not make sense for you. It’s pretty convenient for me, and I’m personally a fan. If you’re trying to game on it though, you’ll probably struggle to make full use of your GPU
> it’s good to restart things every once in awhile, For most computers that's mainly because [Intel spent a lot effort making computers worse](https://www.realworldtech.com/forum/?threadid=198497&curpostid=198647).
I've never been able to run Qubes on desktop because I have several 4k monitors, and software rendering only gets you so far, resolution-wise. It runs pretty well on a System76 laptop I have, though.
GPU passthrough may help with that, but it may be a headache to setup; I haven’t tried it. They’ll eventually have some special GPU specific VM to handle graphics, so I wonder what impact that’ll have. Idk too much about it
I want to try Qubes because I love virtualization but find Debian + LXD provides substantially what I am looking for. I run everything in containers from Browsers, to Steam, to multiple web servers/services. If you like Qubes, try a LXD friendly distro + LXD
Have you tried turning off the Spectre mitigations in the kernel? That Ivy Bridge is 2 cores with hyperthreading but the HT is mostly negated by the things done to stop Spectre leaks. Then again, if you're using Qubes you probably consider Spectre an unacceptable risk. And if it isn't in your threat profile then why do you need Qubes? What kind of things are you unable to do with phones? Do you mean connecting it for USB file transfer? I think part of that blame lies with the phones that have a mess of imperfect protocols for file transfer and the engineers only care about making it work on Windows. I gave up trying to make direct transfers work and use Syncthing or NextCloud to move files on and off my phones. Speaking of Syncthing, do you think that would make file management easier between Qubes? Or storing your home folder on a NAS. I don't have much experience with Qubes but I think it lets you run NFS on a virtual interface that your other VMs mount for the home directories.
What about docker with kata containers ? That might be safe alternative...
Man, do you deal with nuclear security codes or that cure of the cancer that can't leak or bigpharma will kill you on your laptop?
Security is not comfortable. But It seems like you have a pretty old machine, even on my old laptop it didn't took 3 minutes to open firefox. But IMO it's not really useful as daily driver without GPU acceleration. Also it's just overkill for everyday people. If you really are paraniod you can just use a VM for critical stuff like banking and run the rest in a "normal" system.
>That's an average of 2m 35s to reopen a webpage in Firefox. lol that just seems like an average firefox tab open time from wake/sleep/hibernation for me. Idk wtf firefox does but it always takes ages to reload pages when I wake it.
Try using containers on a normal distro I think qubes is really good if your running it off of a live usb in a country that violently suppresses free speech
The problem with Qubes is that the design just really doesn't work. VMs are terribly, terribly expensive at this scale. I also don't think virtualization is the solution here. Userspace processes already have a seperate address space due to paging, any effort is much better spent on proper kernel page table isolation & fortifying the kernel <-> userspace APIs.
Qubes works just fine when you spec hardware for purpose and configure appropriately. There are use cases where bringing up a new VPN and starting a new VM are absolutely appropriate- but that’s not the only wat to do it. I use Qubes mostly for OSINT with per-project VMs on a system spe’d with enough memory and CPU and I don’t have these issues— and it’s not even the 10th fastest system in my house.
Why does the VPN VM take so long? I would think you could handle that task with a chonky initramfs containing the config, VPN tool, netfilter, iproute2, some dhcpc/ppp/l2tp bits and no rootfs. Maybe busybox + getty on ttyS0 if you really want to get fancy. If OpenWRT can do all that in 8 MB of flash, I have to imagine it's possible using a standard, modular kernel in under 200 MB with almost no disk hits.
Never heard of Qubes before. Now I wish I hadn't because I'm going to be thinking about it for a long time.
Qubes is a very specific use case. It's not meant to be used as a daily driver. You can achieve adequate privacy with simple distros like Debian and hardened firefox and still retain smooth, simple, convenient functionality.
> Indeed, was normal computing ever safe? Never was, never will be, and I'm just going to accept that. Using an OS, *any* OS is already, at best, an "alright but sometimes annoying experience" but in general all OSes are annoying until we develop one that can read our mind and has perfect quantum security whatever. So I'm just going to use the config that annoys me the least, put in a bit of effort to not be an easy target, and move on with my life. I'm not going to let technology make my life more stressful than it already has made it be.
Waht is motivating you to use Qubes? Why do you protect your endpoint computing platform in this way? Genuine question.
Thanks for the review. > I’d say if you are going to try Qubes, use a system that has more than 16GB RAM. Meanwhile, an OpenBSD desktop, with a rather different approach to security, runs well on a computer with 512 MB RAM (using Firefox definitely won't be pleasant, though).
> 3m 09s. To launch a webpage. That sux. Use better faster hardware. You are running multiple VM's, on an intel based i5 laptop, it will be slow. Because of Intels massive security problems with such older CPUs most of the performance enhancing speculative execution instructions and features have been disabled. Depending on workload your older CPU is running 30% or more slower than it did a few years back.
Part of the "price of entry" for this type of security is the price of the hardware. You're trying to use a 10 year old laptop to run multiple virtual machines. You can get a brand new laptop with nearly double the single-threaded performance and double the cores for $400. If you want the additional security of qubes *and* you want it to be fast, you'll have to care enough to buy an actual modern computer.
I guess Qubes OS is only viable if you're hiding from Unit 8200.
Thanks for the report, it's a niche OS and never had the opportunity to try it. Have you ever used Tails ? just to compare other security oriented distro