Linux Ubuntu Dual-booting horror

TheCMK , 5 months ago

Imo at this point you're better off just nuking everything (after you backed up your important files) and reinstalling with the following process:

• Install windows, with the linux drive removed
• Plug linux drive back and install ubuntu on it (make sure you select the right drive in the installer)
  If you do this ubuntu should automatically detect windows. If it doesn't you should still be able to boot into either one of them using your device's bios.

Classy OP , 5 months ago

I am going with this game plan, courtesy of ChatGPT 4. It aligns very nicely with my goals and the steps involved also seem to coincide nicely with the suggestions made in this thread. Gives me a path forward, which is really what I need now.

1. Backup Personal Files:

   • Access your Linux partition (A:).
   • Mount the Windows SSD (C:) and HDD (F:) within Linux.
   • Carefully check for any personal files on C: and copy them to F:. Verify that all important data is on F:.

2. Create a Full Backup of F:

   • Connect an external storage device with sufficient space.
   • Create a full backup of your HDD (F:) to this external device. This is crucial to prevent data loss.

3. Verify Backup Integrity:

   • Ensure that the backup on the external storage is complete and the files are accessible.

4. Prepare for OS Installation:

   • Download the latest Linux Cinnamon 23.10 ISO.
   • Create a bootable USB drive with this ISO using a tool like Rufus or Etcher.

5. Remove HDD:

   • Power down your PC.
   • Physically remove the HDD (F:) to ensure it remains untouched during the OS installation.

6. Boot from Linux USB Drive:

   • Insert the bootable Linux USB drive.
   • Access BIOS/UEFI and set the USB drive as the primary boot device.

7. Install Linux Cinnamon on SSD:

   • Proceed with the Linux installation on the SSD (formerly C:).
   • During installation, select the option to format and use the entire SSD. This will erase Windows 10 and all data on it.

8. Post-Installation Checks:

   • After installation, boot into Linux Cinnamon.
   • Check that all hardware components are recognized and functioning.

9. Reinstall HDD:

   • Power down your PC.
   • Reinstall the HDD (F:).

10. Mount and Access HDD in Linux:

    • Boot into Linux.
    • Mount the HDD (F:). You may need to install additional drivers or file system utilities depending on its format.

11. Data Restoration or Access:

    • If required, restore the personal files from the external backup to HDD (F:).
    • Alternatively, if the HDD is accessible without issues, use it as is for storage.

12. Install Necessary Software:

    • Install any Linux-compatible software that you need.

13. Final System Checks:

    • Ensure everything is working as expected.
    • Make any necessary adjustments in system settings, drivers, or configurations.

14. Create a New System Backup:

    • Once everything is set up, consider creating a new system image backup.

Nisaea , 5 months ago

You might not want to trust chatgpt with anything where accuracy and specificity are both essential to not brick a machine or lose data though. Especially since you can't check the sources it used. It did a decent job at getting the general workflow right, but even that's not a given. :/

Kalcifer , 5 months ago

I don't fully understand how you are going about your installation. Are you attempting to install Windows 10, and Ubuntu onto different partitions on one drive, or each on a separate drive? Are you wanting to use grub to boot the OS of choice, or to choose the boot device from the BIOS boot device menu? In what order did you initially install the OS's (Windows 10 first, then Ubuntu, or vice-versa)?

Classy OP , 5 months ago

Dedicated SSD (C:) has Windows on it.

Internal HDD has F: (storage) and a partition for Ubuntu, which is the A: drive in Windows and /dev/sda3 in Linux.

I was originally hoping to do it this way and be able to dual boot them, but the more I think about it the more I feel that just going straight to Linux and biting the bullet would be better in the long run, and I can flash Windows if I really need it.

I was trying to get GRUB to act as boot loader but for the longest time I couldn't even get GRUB running, even with at least 5 different troubleshooting ideas. Then once it was running, I still couldn't get it to mount the EFI. Then I started getting the boot loop issues and Windows OS stopped working, but I figured out how to get Linux desktop running via shimx64.efi in the BIOS boot loader.

This computer is natively a Win 10 machine, and I was trying to add Linux to it.

Kalcifer , 5 months ago

I encourage you to read through this Archwiki page on dualbooting -- it has a lot of very helpful information on the topic.

I was originally hoping to do it this way and be able to dual boot them, but the more I think about it the more I feel that just going straight to Linux and biting the bullet would be better in the long run, and I can flash Windows if I really need it.

It is certainly possible to dual boot (in my experience, with the occasional headache that you may, or may not be willing to deal with) Windows and Linux, but yes, the most reliable installation would be one, or the other.

I was trying to get GRUB to act as boot loader but for the longest time I couldn’t even get GRUB running, even with at least 5 different troubleshooting ideas

Hrm, I'm not sure how you are going about your installation procedure. You mentioned that you are installing Ubuntu, but Ubuntu should come with Grub pre packaged. Installing Ubuntu really only requires clicking a couple buttons in the install wizard.

but I figured out how to get Linux desktop running via shimx64.efi in the BIOS boot loader.

Im not really familiar with shim, but, from what I understand, it's the loader for when you have secure boot enabled, but you should have that disabled.

Classy OP , 5 months ago

Hrm, I'm not sure how you are going about your installation procedure. You mentioned that you are installing Ubuntu, but Ubuntu should come with Grub pre packaged. Installing Ubuntu really only requires clicking a couple buttons in the install wizard.

I had Linux installed on a tertiary partition of a secondary drive, my F: drive. I neglected to store it within the dedicated SSD, C:, and I believe that GRUB was just not being picked up because instead of being sda or something close to it, it was instead sda6.

Further this with the fact that GRUB seemed to not have elevated permissions and when I eventually got into its command line, it was not able to run Linux for reasons I'm unaware. Windows BIOS menu never had Linux or any corollary term available as a boot order item, and only through digging through the Boot from EFI item and submenus was I able to find anything to actually boot Ubuntu without live CD at all. grubx.efi (or whatever it was called) black screened.

Im not really familiar with shim, but, from what I understand, it's the loader for when you have secure boot enabled, but you should have that disabled.

Scout's honor, I really did disable Secure Boot. I did so through Shell. I did so in the Windows directory. I even triple checked that it was disabled through the BIOS menu. If shimx64.efi is only supposed to work when Secure Boot is disabled, that must be because Windows has just been acting screwy as all get out.

Kalcifer , 5 months ago (edited 5 months ago)

I had Linux installed on a tertiary partition of a secondary drive, my F: drive. I neglected to store it within the dedicated SSD, C:, and I believe that GRUB was just not being picked up because instead of being sda or something close to it, it was instead sda6.

First of all sda refers to a physical drive, whereas sda6 refers to a partition on that drive. As for the rest of it, I'm not exactly sure what you are talking about -- it doesn't really matter where in your system Linux is installed; the bootloader probes for an OS, and, once found, will update its table with the position of the OS on the drive.

Further this with the fact that GRUB seemed to not have elevated permissions

This statement doesn't really make sense; Grub runs independently of Linux (it even loads before initramfs), so the concept of "execution privelege" doesn't apply. (source)

when I eventually got into its command line, it was not able to run Linux for reasons I'm unaware.

I will point you to this answer, if you wish to boot linux from the Grub command line.

Windows BIOS

There's no such thing (well, as far as I'm aware, anyways -- maybe a microsoft surface, or the like, labels it as such 😜). The BIOS is contained within a physical chip on the device's motherboard.

Windows BIOS menu never had Linux or any corollary term available as a boot order item

If you're talking abou the boot menu, it doesn't necessarily have to list the linux distro. If you know what drive it is installed on, you select that, then the BIOS finds the bootloader from there. A boot device is just that -- a device to boot from, not an OS to boot into.

I really did disable Secure Boot. I did so through Shell.

I don't understand what you mean here. As far as I'm aware, secure boot is only able to be disabled within the BIOS.

I was unsure if maybe there was some alternative command line trickery that exists to modify it that I am unaware of, but a quick websearch seems to corroborate my pre existing belief.

Windows has just been acting screwy as all get out.

This is an unfortunate reality of dual booting with Windows. Windows can do all sorts of trickery on your system (even when the system is powered down!). If I want to boot Windows (I keep it installed on a separate, dedicated, and air-gapped drive), I plug in its drive, and disconnect all other drives related to Linux. This has been the most reliable method that I have found to dual boot Windows. However, this method is still not without possible issue, as Windows can still leave devices in weird states that end up messing with how they are used in Linux.

Classy OP , 5 months ago (edited 5 months ago)

Thank you for the breakdown. I've learned a fair bit about the infrastructure of my computer and Linux since posting this, and I'm now dedicated Ubuntu with only needing to do some very minor work with WINE here and there in the last two weeks or so. Linux has been a blast and learning about everything has been a lot of fun so far.

My problem turned out to be something with the BIOS. I don't know if a switch got flipped somewhere along the way or what, but when I reset the BIOS to factory default settings in the boot menu I no longer had issues with boot looping and a CPU I could fry an egg on.

I do believe that GRUB was initially installed on sda2 and not sda, and Windows was just taking precidence over grubx64.efi upon startup. Now that I've scrubbed windows from my PC I no longer have any issues booting up and my PC seems to run just about the same as before, less a few graphical funnies with some larger proprietary software I use.

Funny enough, I tried to do a clean install of Debian with KDE on my system and I went back to having boot issues, mainly where it would just open to GRUB CL and I couldn't get it to initialize Debian, when I was certain it was a good install. So I'm just going to stick to Ubuntu for a good while and learn it. Once I feel very confident in filesystem maintenance, command line navigation, snap/flatpak/.deb/whatever, all the major things, I'll start shopping around for another distro again. Ubuntu has been treating me very nicely though.

Kalcifer , 5 months ago

Thank you for the breakdown.

You are very welcome! 😊

I’m now dedicated Ubuntu

A very fair decision! Dual booting can be a huge pain, and, for the average user, it really isn't all that necessary anymore -- Linux has come a very long way!

My problem turned out to be something with the BIOS. I don’t know if a switch got flipped somewhere along the way or what, but when I reset the BIOS to factory default settings in the boot menu I no longer had issues with boot looping and a CPU I could fry an egg on.

Interesting. I'm curious what the setting was. But, I'm glad that it worked out for you in the end!

I do believe that GRUB was initially installed on sda2 and not sda

I refer back to my previous comment -- sda2 refers to a partition on the drive named sda. You could have a drive sda, sdb, sdc, etc. If one was given some partition sdc3 that means it is partition 3 on drive sdc. Everything gets installed into a partition on a drive.

Windows was just taking precidence over grubx64.efi upon startup

This can certainly happen -- especially if Windows is installed after Linux. I woud refer you to this answer to fix it.

less a few graphical funnies with some larger proprietary software I use.

Yeah, I'm not too surprised about that (depending on the speicfic graphical issues that you are referring to, mind you) -- especially if you are using Wine. If you don't mind me asking, what software are you wanting/needing to use?

Funny enough, I tried to do a clean install of Debian with KDE on my system and I went back to having boot issues, mainly where it would just open to GRUB CL and I couldn’t get it to initialize Debian, when I was certain it was a good install.

Hm, this is strange. I would err on the side of a layer 8 error, but there could certainly be some other fuckery afoot.

So I’m just going to stick to Ubuntu for a good while and learn it.

There's no problem with that! Ubuntu was the first distro that I used, as well, when I first got into Linux. Granted, I didn't stick with Ubuntu for long, cause I got mildly annoyed with how it worked.

Once I feel very confident in filesystem maintenance, command line navigation, snap/flatpak/.deb/whatever, all the major things, I’ll start shopping around for another distro again.

Sounds like a solid plan! When you do decide to move on from Ubuntu, I'd recommend Arch LInux 😜

Classy OP , 5 months ago

I refer back to my previous comment

Yes, I am aware of how the partition naming structure works, to a degree. I am going off the fact that when I installed Ubuntu, it was installed on a partition (sda2) rather than a primary drive (sda). I've read that when GRUB is installed, if it gets installed to /dev/sda2 rather than /dev/sda it can cause issues with dual booting as the BIOS will read in a sequential order, and it may miss a partition if it's "far enough down the list". As another example, you may be in for some trouble if grubx.efi is installed on /dev/sda8 or something.

Everything gets installed into a partition on a drive.

I guess I must have gotten my preconceptions wrong, or perhaps I misread something. From my impression, certain things can be installed on the primary drive such as boot loaders, but I could be wrong.

If you don't mind me asking, what software are you wanting/needing to use?

Finale 2012c is the main software I needed. And by funnies I mostly just mean that it's slow to update graphics, but the program works entirely as intended. MIDI drivers work, sound libraries (Garritan and ARIA Player) function, print to PDF is fine, I'm actually incredibly impressed! I'm using WINE 8.0.1.

I also wouldn't mind trying to get Cakewalk running, as my workflow is definitely more attuned to that software, but maybe trying to get all the proprietary drivers (e.g. TASCAM's interface drivers) to work with Linux may be more headache than its worth.

Arch

I've heard it can be pretty challenging to get into Arch, is this true? I don't know if I'll ever be a "script kiddie" as it were. I plan on getting good at using bash and learning the other ones like ssh, but I don't know how much I like the idea of having to hand-craft my OS from bare metal.

Kalcifer , 5 months ago

when I installed Ubuntu, it was installed on a partition (sda2) rather than a primary drive (sda)

The exact meaning of the language in use is somewhat context dependent. It is technically possible to use a block device (e.g. /dev/sda) [source] as a filesystem, but it is generally discouraged -- afaik, this is generally because of compatibility reasons. As to the meaning of a statement that looks something like "Install Ubuntu to /dev/sda" this could be interpereted as essentially just rewriting the existing partition table that exists on that drive with a new one, where, for example, partition 1 (e.g. /dev/sda1) is for the boot partition, and partition 2 (e.g. /dev/sda2) is where Ubuntu lives. In that example, technically Ubuntu is only resides in /dev/sda2, but, for the whole installation process, the user can interpret it as essentially installing it all to /dev/sda.

I’ve read that when GRUB is installed, if it gets installed to /dev/sda2 rather than /dev/sda it can cause issues with dual booting as the BIOS will read in a sequential order, and it may miss a partition if it’s “far enough down the list”

It's worth understanding the boot process of a system (this is more taylored to an average Linux system, but can be generally applied, if one is careful):

1. The machine powers on
2. The BIOS chip on the motherboard comes to life, it gets copied into RAM, and the CPU starts executing it.
3. It finds the first device in the BIOS boot list
4. It looks at the first sector (512 bytes) of that drive (this generally only applies if the drive uses MBR, and can be a little bit different with GPT, but the general process is pretty much the same, afaik), which contain the location of the bootloader on that drive, and copies it to RAM at address 0x7C00
5. The bootloader (e.g. Grub) springs to life and it takes over the boot process from the BIOS
6. In the case of your average linux installation, Grub will then initialize something called the "initramfs" which is sort of like an extremely small Linux OS that gets loaded into RAM
7. Initramfs essentially bootstraps the actual Linux distro into booting -- this is required as booting the desired Linux distro may depend on things that run on Linux which can't exist before Linux is loaded (e.g. LVM's, LUKS encryption, etc.).
8. Now that the OS is loaded into ram, it boots, and the process is complete.

So, back to your statement, the actual program of Grub could reside in /dev/sda2, but the "bootloader bootsrapping" program, which resides in the first 512 bytes of the disk, could be thought of as being installed to /dev/sda.

[source], [source], [source], [source]

As another example, you may be in for some trouble if grubx.efi is installed on /dev/sda8 or something.

The only real "hard" limit on the location of Grub is that, in the case of MBR, it necessarily must be located within the first 2.2 TB of the disk.

[source], [source], [source], [source]

I guess I must have gotten my preconceptions wrong, or perhaps I misread something. From my impression, certain things can be installed on the primary drive such as boot loaders, but I could be wrong.

As I outlined above, this is sort of a technicality in language that depends on context.

Finale 2012c is the main software I needed.

I'm not sure if this is exactly equivalent to that software, but perhaps you would be interested in MuseScore -- it's open source.

I’ve heard it can be pretty challenging to get into Arch, is this true?

This has been somewhat exaggerated through memes by the community, and strange elitism. It's a bit tough to separate oneself from their curse of knowledge, but if one possesses the motivation to learn, it's really not that complicated. Depending on one's existing knowledge, it may initially appear daunting, but the community is quite good, from my experience, and the Arch Wiki is extremely useful. Installation is essentially a matter of just following the installation guide step-by-step.

I don’t know if I’ll ever be a “script kiddie” as it were.

Imo, arch has nothing to do with that. If one wants to be a part of that then prob lurking around the Kali Linux communities would be a start. Do note that I am not speaking about Kali Linux from experience, just hearsay, so take that with a grain of salt. But, yeah, Arch is more for people that want more fine-grained control over their system without wanting to get into the full-time job that is something like Gentoo 😜.

I don’t know how much I like the idea of having to hand-craft my OS from bare metal.

Imo, that's not really what arch is -- even Gentoo isn't like that. The closest to that would probably be something like Linux From Scratch. Arch just gives you more freedom to choose the base software that your system is using -- stuff like your DE, your networking utils, display server, audio server, etc.

I would like to emphasize that this kind of choice exists with virtually all Linux distros -- as in you can essentially make any distro "look" like any other (there may be some intricacies that I am unaware of that may get in the way of changing some things without having to alter others); Arch Linux simply gives you most of the choice right up front.

Classy OP , 5 months ago

It's always a heartwarming experience seeing someone passionate about a subject enough that they'd be willing to dedicate what was likely at least twenty minutes of their own free time to writing a detailed response to a stranger on the internet.

re: /dev/sda haberdashery

Your explanation was very helpful in explaining the process by which the BIOS is loaded. As I've continued to work on Ubuntu, I've been trying to hammer out little errors along the way and I believe that I inadvertently identified the problem with my dual-booting situation before. Whenever I load Linux, the system will load that ubiquitous screen where it does a filesystem check, etc, and I always get two errors: (1) VMX (outside TXT) disabled...; (2) ima: error communicating with TPM. I went into the BIOS and figured out how to turn the TPM on, and when I did so... what do you know, I started boot-looping again, just as before. Apparently I'm going to have to do a bit of troubleshooting to get Linux operable with the TPM, if I care enough about it to just undo a simple error message on boot-up that has no impact on my actual computing experience. But having his TPM chip before was causing boot-looping, perhaps due to a security issue with grubx, who knows, but for the time being I'm putting it on the back-burner.

re: Musescore

I appreciate the thought, and yes Musescore has been on my periphery a good percentage of my 15 years of using notation software. Musescore is an admirable project and I'm impressed with the steps its taken in the last few updates. Frankly, this has probably been the fourth or fifth time now that someone has hocked Musescore as a FOSS alternative to Finale, and while I get it, they are not truly one-to-one in compatibility, at least not yet. Finale is a boutique program, designed for professional use and it's very feature-rich, especially as one gets into more specialized concerns in terms of unusual notations, etc. Finale works just fine on my system and I don't intend on changing away from it anytime soon. I've been using it for so many years, it's like second-nature to me. I couldn't imagine dropping a software I spent hundreds of dollars on now for something else if I still get great mileage out of it.

re: Arch Linux

Following the last time you and I communicated, I actually saw a video from SomeOrdinaryGamers where Muta did a step-by-step installation of Arch on a new machine. It certainly seems more complex than Ubuntu, but at the same time, boy does it give you a rich experience in learning the intricacies of your system and how everything functions together. I am definitely going to be keeping Ubuntu on my main system for the time being, but I do have a blank ZBook15 gen 2 (I believe it has Mint on it right now? I haven't opened it in a few months...) and I might have a go at installing Arch on it and messing around for a while.

My current project is going to be taking my secondary HDD, which is only a storage device now, and configuring its file structure to be easier to use with Linux, as well as clearing out all the legacy OS files from when it used to have Windows on it. I've been having trouble using utilities like rm -rf because for some reason, some files will delete with no issue, but then others will actually cause the drive to crash in some spectacular fashion, and I have to sudo umount -l then remount again with ntfs-3g just to get back to it. I can't tell if its a permissions issue or something else. I know the drive is old and there are four damaged sectors, but the most recent SMART test didn't seem to throw up any major red flags. I can delete individual problem files, but trying to delete a bulk quantity runs into issues at times. It's weird. I don't exactly want to format the drive because there's ~0.9TB of personal files on there (that are all backed up both on a cloud service and an external SSD, no worries!), but so far I'm having fun learning some new commands.

Kalcifer , 4 months ago

It’s always a heartwarming experience seeing someone passionate about a subject enough that they’d be willing to dedicate what was likely at least twenty minutes of their own free time to writing a detailed response to a stranger on the internet.

❤️

(1) VMX (outside TXT) disabled…;

From what I can see, this isn't that big of a deal. It's just a warning (technically it is an error, but, essentially it's a warning), that Virtual Machine Extensions aren't enabled in the BIOS. Unless these are required for the boot process of your distro (which I seriously doubt), it shouldn't cause you any problems, unless you explicitly require their functionality for some other program.

(2) ima: error communicating with TPM. I went into the BIOS and figured out how to turn the TPM on, and when I did so… what do you know, I started boot-looping again, just as before.

That's... strange. Are you certain that it isn't the converse? Very strange that enabling the TPM would cause issues. It could certainly make sense for it to cause issues if the TPM was in use, and it was disabled.

In case you are unaware, the TPM is essentially a chip on the motherboard (in most cases, anyways -- it potentially could be in a different form within the CPU e.g. fTPM)

Apparently I’m going to have to do a bit of troubleshooting to get Linux operable with the TPM

It's completely possible, and I would certainly not discourage it's use -- especially if the device is a laptop. It's, of course, not the be-all and end-all of security, but full disk encryption with a TPM is definitely a good first line of defence. Obviously, it's better to manually input the encryption password, rather than having it be released by the TPM, but I certainly wouldn't blame someone for opting for the more user friendly alternative.

Finale works just fine on my system and I don’t intend on changing away from it anytime soon. I’ve been using it for so many years, it’s like second-nature to me. I couldn’t imagine dropping a software I spent hundreds of dollars on now for something else if I still get great mileage out of it.

For sure! If you are comfortable in your work flow, then by all means stay wiht it. Ethically, though, it of course doesn't hurt to keep FOSS alternatives to proprietary software in the back of your mind 😊.

It certainly seems more complex than Ubuntu

It sort of depends on exactly what you mean by that, but I certainly wouldn't argue with the colloquial statement that it's more "complex" -- especially the installation.

but at the same time, boy does it give you a rich experience in learning the intricacies of your system and how everything functions together.

Absolutely! And if you want to go another step further in that understanding, then I would recomend looking into Gentoo.

but then others will actually cause the drive to crash in some spectacular fashion

I think I may know what you are talking about with this. I have experienced issues with external HDD's going to sleep when they are being read from, or written to, but I attribute that to USB sleep modes. So, if you are referring to an internal SATA drive, then I'm not sure what would be causing that issue.

and I have to sudo umount -l

I would caution against using the -l/--lazy flag. It may present you with unintended consequences. It woudl be better to find what process is keeping the dive busy before attempting an unmount.

[source]

then remount again with ntfs-3g

Out of curiosity, is there any particular reason why you're using the userspace NTFS driver, rather than the included kernel driver?