disk partition for the head-start: dive into the partitioning of the HDD

[d-hubs]

 

g day dear Friends, 

 

 

well I have a new and Fresh (that means clean)  HDD in my laptop on which I want to install EndeavourOS (the newest release - derives in Feb i Guess)
So, as I not tooo many  (in fact really not too much ) knowledge on Linux File System hierarchy,  I thought some would be helping me to choose the best one.  Here is what I briefly do on Notebook.

i want to dive into the topic of programming (should maintain my project files)  That said: Sometimes, I need to do some installations too  that means install few external packages too VSCode etc.etx. Jupyterstuff etc.

 

 Though they never were more than 300 MB each i guess) 

 

• I use software's like VSCode, etc.
• Store lots of music, photos
• I have a good set of ebooks, office documents (includes documentations of packages).
• Download many files..and manuals as pdf., 
• This is what a normal user would probably do.

 

What I am actually searching for is that, how do I maintain all those file if I have created only 3 partitions "/" and "/home".

Of course, all of these would come under "/home". Wouldn't they look messy??

so if it comes ot a  disk partition for my notebook: i need some ideas: Well that said: this is due to the lack of knowledge:

 

Guys - i want to know how to format a storage drive from the terminal.

a very helpful thing would be to dive into all that steps - and the options for commands and to get a base knowledge that i (hopefully) am able to use to extrapolate future uses.

as said above - i eagerly would like to know how to format in the different file systems such as NTFS, FAT32, EXT4:

 

These common formats are something that i think are the ones that are used in many many use-cases: So i think i have to get more information on how to partition the drive via terminal.

besides the notebooks i mentioned above - i sometimes want to set up a a high capacity external hard drive (HDD) from the terminal. Well - my friends often told me, that I can use gparted for this job as well as other GUI programs, Sure thing. but I still want to now how to do it from the terminal. I think that the terminal is damned powerful - and i want to learn to do more on this powerful tool.

 

So i conclude: it would be awesome if i know, how to format a storage drive from the terminal.,i want to know how should I partition my HDD and what directories should I mount them to those partitions and  finally: what kind of files should I place in them.

 

Any and all help is greatly appreciated 

 

 

[Kilrah]

-> Moved to Linux, macOS and Everything Not-Windows 

[OddOod]

36 minutes ago, d-hubs said:

HDD

That's a bold choice for an OS drive. You sure you want to do that?
 

37 minutes ago, d-hubs said:

What I am actually searching for is that, how do I maintain all those file if I have created only 3 partitions "/" and "/home".

There really should only be three partitions: EFI/boot, swap, and primary. You *do not* touch those first two. The third is where you store all your data. 
If you absolutely must separate the OS from the data partitions, you can, but very not necessary. 

 

40 minutes ago, d-hubs said:

Of course, all of these would come under "/home". Wouldn't they look messy??

The messiness of file storage is mostly down to how well you lay out your directories. Keep stuff organized and it's not messy.
 

40 minutes ago, d-hubs said:

Guys - i want to know how to format a storage drive from the terminal.

https://man7.org/linux/man-pages/man8/parted.8.html

[y0ur5h4d0w]

about your request: (the explanation i give is simple just to give you an idea about what is, it might not be 1000% technically perfect)

boot partition, same as windows actually but different bootloader
swap, optional in some cases, depends on the quantity of ram you have

primary, well, self explainatory...

home partition, it's nice to have if you want to be safe but it's totally optional, these days with all the snapshot and backup software is not so necessary to divide it from primary

 

 

the mess you mention simply depends on how you move your data and how you store it inside your home folder, pratically speaking, the home folder is the exact same concept of the "c:\users\YOUR_USERNAME" folder in windows, how you populate is arbitrary. 

 

 

after this, a different disk is reccomended for the backups/snapshots, in a nutshell this should be almost everything

 

all this being said, if you don't have much base knowledge and you want to try linux to get your feets wet, just install ubuntu and explore your fresh install by hanging around the folders

 

this is a little but actually good start point >>> https://cyberpanel.net/blog/linux-directory-structure

[Ralphred]

5 hours ago, y0ur5h4d0w said:

boot partition, same as windows actually but different bootloader

You need to be careful about describing "a boot partition", having /boot as a separate partition is no longer necessary for UEFI, all you NEED is a small fat32 (128meg is enough for most use cases) formatted EFI partition, with the type set to "efi" by your partitioning software, where it gets mounted is mostly irrelevant as when issuing grub-install commands you can use the --efi-directory=<location> switch, and you only really have to do that once in a blue moon.

 

5 hours ago, d-hubs said:

i eagerly would like to know how to format in the different file systems such as NTFS, FAT32, EXT4:

mkfs.ntfs , mkfs.fat -F32 , and mkfs.ext4 , all followed by /dev/<target partition>

 

Most distro's ship a gpt enabled fdisk nowadays, and it's easier to use than parted.

5 hours ago, d-hubs said:

I still want to now how to do it from the terminal

The "correct" way would be to have the UUID of the external drives filesystem in fstab, set to mount in /mnt/myExternalDrive, but with the "noauto" option set so things don't complain when it isn't plugged in, and the "user" option set you you can mount and unmount it without having to escalate privileges. This way you can just issue a "mount /mnt/myExternalDrive" command and it gets mounted, then "sync && umount /mnt/myExternalDrive" to prepare it for being unplugged. You can also create a symlink to the mount location in your /home/{Documents,Desktop,whatever} directory with "ln -s /mnt/myExternalDrive /home/user/Desktop/ExternalDocs" or similar.

 

You don't mention if you will be dual booting, but you can also mount an NTFS windows drive as /mnt/windows (be sure to set the type as ntfs3 in fstab), then use the --bind option with mount to replicate your windows home directory structure in your /home/user directory, for example your fstab could contain:

UUID=1234567890ABCDEF                 /mnt/windows           ntfs3 rw,noatime,iocharset=utf8 0 2
/mnt/windows/users/myuser/Desktop     /home/myuser/Desktop   none  defaults,bind             0 0
/mnt/windows/users/myuser/MyDocuments /home/myuser/Documents none  defaults,bind             0 0
/mnt/windows/users/myuser/Videos      /home/myuser/Videos    none  defaults,bind             0 0

Bind mounts can be displayed a little oddly in /etc/mtab and /proc/mounts, so be ready for that.

 

EDIT: A general tip for keeping storage drives tidy is to always mount them in /mnt/something, then create a directory that describes it's content there (e.g VideoClips) then symlink it to the location you want to find video clips with ln -s /mnt/storage/VideoClips /home/myuser/Videos/Clips

[NoLeafClover]

There is no "perfect" layout. A lot of it depends on your personal preferences. As others have mentioned the most basic layout contains:

1. EFI System Partition (ESP)
2. /boot partition - traditionally, or just use the ESP as /boot is no longer strictly required on an EFI system
3. / - the root (OS) partition
4. Data storage partition - could be mounted as /home or elsewhere

Data should absolutely be separated from the root (OS) partition as "best practice", e.g. reinstalling of the OS does not interfere with your data.

 

Obviously, there are nuances with increasing complexity:

• If you envisage the possibility of running out of RAM, you might need a swap partition.
  • This can also be configured as a swap file instead, so it's not critical if you don't have one at the start. A swap file has the advantage of being easy to adjust its size.
• If you envisage the possibility of adding more "partitions" to better organise your data, you might want to consider formatting one partition as LVM and making multiple virtual partitions within the LVM container.
  • This has the benefit of making re-sizing of the partitions relatively easy, unlike "physical" partitions which is often far more tedious to deal with.
  • Note that some filesystems also allow for virtual volumes on a filesystem level, I believe btrfs is one of them (I haven't yet used it so not sure).
• If you want to make use of disk encryption then things become a bit more complicated:
  • For just the data partition, you might want to format that as a LUKS (Linux Unified Key Setup) encrypted volume, and a filesystem within it.
    Some filesystems support encryption on the filesystem level, but LUKS can provide you with better "zero leak" protection as not even the underlying filesystem type will be exposed without unlocking the partition first.
  • If you want to encrypt the OS itself as well, then you have two options:
    • format as LUKS volume first, and then the filesystem within - this has the downside is it you will have to unlock two partitions separately.
    • have a LUKS formatted partition, an LVM container within, and virtual partitions inside the LVM container.

As an example, a layout of an NVMe drive identified by "/dev/nvme0n1" that takes advantages of all of the above and includes the [optional] "/boot" partition could look something like:

/dev/nvme0n1
├─/dev/nvme0n1p1: EFI System Partition
├─/dev/nvme0n1p2: /boot
└─/dev/nvme0n1p3: LUKS Encrypted
  └─LVM Volume Group
    ├─LVM Volume: /
    └─LVM Volume: /home

The layout above shows only 3 "physical" GPT partitions. The 3rd one is an encrypted container, within which there is a Logical Volume container, within which there are two Logical Volumes (or virtual partitions) - one for "/" and one for "/home". If you don't want a "/boot" partition you will need to make sure the ESP partition is large enough to store at least two kernels and their associated init images (1 GB should suffice).

 

I would strongly encourage something like the above, especially on a laptop that can be at risk of being lost or stolen.

 

It sounds complicated and, unfortunately, in many cases tuning it exactly the way you like may require a bit of manual intervention. But since you asked about doing things 'the CLI' way, this could be something to explore. If you're new to Linux, I would recommend that you start simple - forget about LUKS and LVM - and work your way up. Use disposable virtual machines to play around and get familiar with how to do it. It's not complicated, but may seem very intimidating the first time and familiarity is key. There's a lot to absorb here, I know. But this is part and parcel of the amazing flexibility that you have on Linux. And, if you're the tinkering type, practice makes perfect.

 

As to organising files... that is something you'll have to figure out yourself. Find what works for you. But keep in mind that if you have a habit of dumping things in "/home", even if you later re-organise them, then you most certainly will need to make sure "/home" is on a separate partition or allocate enough storage to "/" so that you don't run out of space on "/" which could interfere with system updates.

 

Bind mounts, as mentioned by Ralphred, are a good way to "alias" this. Think of them as a folder shortcut on Windows, except it doesn't appear as a shortcut and behaves just like a regular folder. You can, for example, create a directory on "/mnt/your-storage-partition/home" and bind it to "/home". Thus everything you put under "/home" will effectively be stored under "/mnt/your-storage-partition/home". Ralphred's "MyDocuments" directory is an example.

 

Bind mounts are also very versatile for storing things that otherwise would go on your root partition somewhere else. An example of this is Flatpak applications. They normally go under "/var/lib/flatpak" and these can quickly fill up your root partition, depending on its size. A good way around this is a bind mount such as:

# /etc/fstab
# Syntax follows this pattern:
# <source>          <mount point>       <filesystem>    <options> <dump> <pass>
/mnt/data/flatpak   /var/lib/flatpak    none            bind      0      0

Personally, I don't like having a large OS partition - it's pointless. In 20y+ of using Linux, while requirements have sloooowly grown, I find a 30GB root partition is enough for all of my needs and I have rarely gone over 60%. For isolated cases, like Flatpak, I just use a bind mount which also has the benefit of making the size requirements for an OS partition more predictable.

 

Symbolic links (symlinks), also mentioned by Ralphred, are also very handy. They are more akin to the Windows shortcuts as they present themselves as such, and can simply be created or deleted like any other file.

[Guest]

9 hours ago, OddOod said:

If you absolutely must separate the OS from the data partitions, you can, but very not necessary.

I strongly disagree.  The OS (and most of userland) is largely "read only" -- with the exceptions of logs and configuration files.  OTOH, "data files" tend to be R/W.  By separating them, you can let the system give you additional protections on their integrity and recoverability.

 

E.g., having to fsck a TB disk takes considerably longer than a 100MB disk.  And, is likely going to turn up more issues.  Will you know which, if any, of these affect the system and its usability?

 

I run *BSDs with a typical partitioning of:

• /       smallish (less than a GB).  Even smaller if I could discipline myself to stop working as root (which leads to me playing in /root all the time)
• /var  also smallish (similar to the above).  It holds machine generated log files.  If I really want to have something big accessible under /var (like a DBMS store), then I can hard/sym-link it to some other area with more space.
• /usr  modestly sized (typ about 3G) for most of the "standard" userland
• /usr/pkg  also modestly sized (2G?) for the "optional packages" that I typically install
• /home  reasonably large (20G for a few user box) for the various real and virtual users of the system (typically just me -- under many aliases)

If there is a problem with the disk (power outage during some heavy activity), the chances are that / will be intact -- because nothing ever really gets written/updated, there (besides the occasional SYSTEM configuration changes in /etc).  So, I can boot to a usable machine, even if in single user mode.  From there, examine the remaining partitions to see how readily they can be brought back online.

 

I typically add /0, /1, /2, ... for other mounts, external drives, etc.  As I tend to build/rebuild kernels and other system software, I create /Sources and symlink into it from other places in the legacy hierarchy (e.g., /usr/src, /usr/pkgsrc, /usr/xsrc, etc.).  And, /Playpen as an extended /home (with symlinks into it for various virtual users representative of new projects:  /home/Model3, etc.)

 

In this way, I can replicate a machine just by tarring a selection of top level directories onto a similarly partitioned "new" machine/disk.  The changes that are likely going to be made (wrt usage or partitioning) will happen outside of the legacy hierarchy so I can have a new machine up and running in a matter of minutes; the bulk of the configuratiuon changes residing in the /etc/rc.conf text file.

 

[Ralphred]

14 hours ago, serverfarm said:

By separating them, you can let the system give you additional protections

It also gives you the ability to mount certain areas using the "noexec" option, but these sorts of considerations are really "server security practices".

 

I would like to tinker with the idea of mounting /home as noexec, as an exercise in defeating "user-error related malware", but it feels a bit like using a sledgehammer to kill an ant...

[Guest]

8 minutes ago, Ralphred said:

It also gives you the ability to mount certain areas using the "noexec" option,

You can also mount file-systems read-only, impose per-user quotas on individual file systems, etc.  This is helpful when someone starts a rampant process that tries to gobble up all disk space in $HOME or tries to scribble someplace where you should have established better ACLs.

 

If you are clever, you can build a tiny, "more complete" (e.g., maybe including a few /usr tools) system on / and then mount the other file systems on top of it; this gives you access to those few additional tools you've opted to include on the "/ system" (when their normal locations aren't yet mounted) while letting the full complement appear when you "mount all".  (e.g., being stuck with just ed(1) on / can be annoying)

[NoLeafClover]

33 minutes ago, Ralphred said:

It also gives you the ability to mount certain areas using the "noexec" option, but these sorts of considerations are really "server security practices".

 

I would like to tinker with the idea of mounting /home as noexec, as an exercise in defeating "user-error related malware", but it feels a bit like using a sledgehammer to kill an ant...

I've not personally tried "noexec" with "/home" for the same "sledgehammer" reason, but overall I have my data volumes always mounted with "nodev,nosuid,noexec" and I keep a separate volume for "executables". I also tend to avoid putting too much in "/home", which is why I keep it small, to encourage myself to exercise some data hygiene and organisation. Probably little benefit overall, but having the no-flags doesn't hurt either.

 

There's also crazy stuff one can do with SELinux, but... maybe too much of an overkill at times for a home config. Except when I get the urge to scratch an itch

[Ralphred]

1 hour ago, NoLeafClover said:

There's also crazy stuff one can do with SELinux

Indeed. I've implemented it twice, and am getting a better understanding, but still can't leave permissive mode because of the all the audit failures that would gimp the machine if I did. I'll "finish" it one day.

 

1 hour ago, serverfarm said:

being stuck with just ed(1) on / can be annoying

Meh, ed is great email client 

[wasab]

I just create one partition, the root /. I like to clean install rather than dirty install my os so I don't need the benefits of mounting my boot or my home in separate partitions which is pretty much the benefit of keeping your home, kernels, ect intact when you need to wipe/format the rest of your hard drive.

 

With that said, you may want to use LVM if you do want different partitions. It is make it a lot easier to extend and shrink partitions in the future. Just don't combine multiple drive into a single logical volume, it is a bad idea which I had to learned painfully. 

 

There is also no issue keeping everything inside your home. In fact this is what I do and what I suggest everyone do. Create a bin dir in your home, add it to path, and throw all your userland softwares in there if you have the option to choose install directory. 

 

Much easier to manage compare to softwares, scripts, and files that are install to /lib, /bin, /usr/bin, /use/local/bin, /opt and ect ect ect like what package managers usually do. Downside is you probably need to manage these softwares versions and uninstaller yourself if they do not come with an internal updater and uninstaller.  

 

 

Pretty sure there is a reason and conventions to have so many different file directory that seem to be doing the same thing but I honestly don't care to learn the how's and whys. It is indeed messy but home dir is a place where I can have total control without worrying about messing anything up. 

[Thomas53]

Having used Linux since just after Linus released it, I find that almost any distribution is intelligent enough by itself to lay out the partitioning system by itself, and the only real requirement is to decide if you want a sleep and/or a hibernation partition.

 

I said almost, because there are some specialty distributions that are not for general usage and do not allow for some of the following:

 

Also, all of this is for a single drive. for multiple drives, I place the home partition on the second drive.

 

After installation. you may want to add a second home partition for someone that will be using the computer often, this will allow each of you to be able to store data that you don't want the other to have access to. 

 

Or if you have young children or need to allow access to multiple people,you can create a guest account which will allow them access to any application installed but they will not be able to make any changes that require root access nor will they be able to save any data to the drive itself, but they can save it on an external device, like a USB stick.

 

IF you need to share the drive with Windows, due to it's limitations, Windows MUST be installed first and afterwards when you install Linux, it will take up as much of the free space as practical, still leaving open space for more data to be written to the Windows OS. 

I'm not familiar enough with MacOS to know if it has the same limitations, but the final results should end up the same.

[NoLeafClover]

16 hours ago, Thomas53 said:

I find that almost any distribution is intelligent enough by itself to lay out the partitioning system by itself

I strongly disagree with this statement for anything more than a basic partition layout.

 

I have yet to encounter an installer of any distro that is capable enough to achieve the layout I outlined above without significant battles and time wasting. In fact, my only real gripe with the state of Linux today is the terrible partitioning in GUI installers. For the most basic of layouts they work, yes. But as soon as you start to introduce a tiny bit more complexity things go tits up real fast.

 

If you want something like the above, the only reliable option I have found is to do the actual partition layout, including set up of LUKS and LVM, manually with the CLI. Once in place, most modern installers will then be able to detect what's going on, usually the "Advanced partitioning" option, and then becomes a smooth sail after, culminating on "use this for A, that for B", etc.

 

Luckily, it's usually a "do once and forget" kind of thing.

 

16 hours ago, Thomas53 said:

you may want to add a second home partition for someone that will be using the computer often, this will allow each of you to be able to store data that you don't want the other to have access to

I may be misreading this. But partition layout has nothing to do with permissions. Permissions are generally pretty savvy. There's no need for a separate partition to "isolate" a second user. Most distros will default users' home directories to "750" (or "rwxr-x---") permissions with each user belonging to their own group. "everybody else" permissions are not given by default, so they won't be able to access each other's home directories at all.

 

Having a separate partition, on its own, will achieve nothing unless the permissions on the filesystem itself are also sane to prevent somebody else from accessing the files.

 

The only benefit I see here is storage isolation. With independent partitions, each user will have their own dedicated storage of up to the size of the partition, and they won't be able to eat into each other's storage if one uses more. Even then, arguably this is more effectively done with filesystem quotas instead as they can be enforced and changed dynamically by a privileged user, unlike partition resizing which, even with LVM, is more faff than it needs to be for this use case.

[Thomas53]

I think I misunderstood what you want to do.

In the Home partition, Linux divides the partition into Desktop, Downloads Music, Pictures, and Video and a few others, mostly hidden, which the average user usually doesn't often - if ever access directly..

 

All or the named ones are only a few megabytes in size and are expanded as needed. The only limitation is the size of the disk itself.

 

You can even add other directories to your home partition or further subdivide the present ones to further separate them. In my Music directory I have it divided by Artist and in my Documents directory, I have it divided my subject, i.e. Tech , further divided by major topic and books divided by subject i.e. Fiction Non-Fiction and then by series or author.