Linux from Scratch

[Grabinio]

Hello,
I apologize for this long thread, I am a lost sheep that needs guidance.

 

Background

I am a senior student studying for my Computer Engineering BE Degree. It is obligatory where I study that students do what is called a (Graduation Project) which is a project you have to complete in two terms period.The project can be whatever you see fit as long as you can secure a supervisor who sees that your suggestion or idea is worth the effort or it will improve the student knowledge, most of the students here develop certain applications that do a certain job for example ( a medicine reminder).

I thought about creating a Linux distro. I think it at least requires more work and more attention. Also I will learn a lot about Linux which I personally think is important nowadays.
The issue is that I have zero experience in Linux development, it is not something you take in a course in uni, I became interested after taking Google Automation Course. So I have no idea where is the limit as it is something you find whilst building the distro I plan on using Linux from Scratch book as a guide. But of course not duplicating the same thing.

Problem

I proposed this to my professor  with the intention of adding software that students who study here might use the issue is:

 

• No clear goal
  My professor states that the idea in general is OK, but the end goal is not clear.
• What difference will this make?
  I explained that basically most distros are the same, it is just different packages and looks.
   

Question

 

I do not think I can specify what I can modify or what I can create by myself or what I can reuse as I said I have never done this before the only why I can find this is whilst doing it. Perhaps a certain software needs a huge team to work on I cant say that I will modify it or create something similar then fail to deliver.

I do not want to change the project but I have to convince my supervisor.
My question is there any suggestions on what I can change or what is impossible to change as a team of 2 individuals ?. For example can I create a package manger?.
I really want to do this from scratch but in order to define a real clear  goal I have to know what I can or cannot modify before I even start working on it. Can someone clarify this for me or do you think that I should just drop this whole idea and look for another project?.

Thank you.

 

[Radium_Angel]

2 minutes ago, Grabinio said:

Thank you.

By and large, re-mixes of existing distros will require a small team of talented folks to work on.

Creating your own *nix and package manager from the ground up, well..I hope you are a kick-ass programmer.

You can start at www.distrowatch.com and dig around, though you *might* be interested in this specific linux

https://distrowatch.com/table.php?distribution=scientific

 

In particular, read who developed it and why.

 

I think creating a custom PM (package manager) will be beyond your time frame at University, but a remix of an existing distro with student engineering in mind might be in the realm of possibility for you.

 

Either way, you are in for a lot of work. Good luck.

[mahyar]

6 minutes ago, Grabinio said:

Hello,
I apologize for this long thread, I am a lost sheep that needs guidance.

 

Background

I am a senior student studying for my Computer Engineering BE Degree. It is obligatory where I study that students do what is called a (Graduation Project) which is a project you have to complete in two terms period.The project can be whatever you see fit as long as you can secure a supervisor who sees that your suggestion or idea is worth the effort or it will improve the student knowledge, most of the students here develop certain applications that do a certain job for example ( a medicine reminder).

I thought about creating a Linux distro. I think it at least requires more work and more attention. Also I will learn a lot about Linux which I personally think is important nowadays.
The issue is that I have zero experience in Linux development , it is not something you take in a course in uni , I became interested after taking Google Automation Course. So I have no idea where is the limit as it is something you find whilst building the distro I plan on using Linux from Scratch book as a guide. But of course not duplicating the same thing.

Problem

I proposed this to my professor  with the intention of adding software that students who study here might use the issue is:

 

• No clear goal
  My professor states that the idea in general is OK, but the end goal is not clear.
• What difference will this make?
  I explained that basically most distros are the same, it is just different packages and looks.
   

Question

 

I do not think I can specify what I can modify or what I can create by myself or what I can reuse as I said I have never done this before the only why I can find this is whilst doing it. Perhaps a certain software needs a huge team to work on I cant say that I will modify it or create something similar then fail to deliver.

I do not want to change the project but I have to convince my supervisor.
My question is there any suggestions on what I can change or what is impossible to change as a team of 2 individuals ?. For example can I create a package manger?.
I really want to do this from scratch but in order to define a real clear  goal I have to know what I can or cannot modify before I even start working on it. Can someone clarify this for me or do you think that I should just drop this whole idea and look for another project?.

Thank you.

 

TLDR you can do whatever you want as long as you call it linux

[eeeee1]

5 minutes ago, Grabinio said:

No clear goal
My professor states that the idea in general is OK, but the end goal is not clear.

find something that you think people need. Some people need really light distros or need server distros. find something that you are interested in making and what people need

6 minutes ago, Grabinio said:

What difference will this make?
I explained that basically most distros are the same, it is just different packages and looks.

I mean thats like asking, what difference did TempleOS make?

7 minutes ago, Grabinio said:

For example can I create a package manger?

sure.

[eeeee1]

Just now, mahyar said:

TLDR you can do whatever you want as long as you call it linux

I'd just like to interject for a moment. What you're referring to as Linux, is in fact, GNU/Linux, or as I've recently taken to calling it, GNU plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities and vital system components comprising a full OS as defined by POSIX...

 

[mahyar]

Just now, eeeee1 said:

I'd just like to interject for a moment. What you're referring to as Linux, is in fact, GNU/Linux, or as I've recently taken to calling it, GNU plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities and vital system components comprising a full OS as defined by POSIX...

 

i said TLDR i know all of that and by calling it linux i mean that to say its based on linux kernal

[eeeee1]

1 minute ago, mahyar said:

i said TLDR i know all of that and by calling it linux i mean that to say its based on linux kernal

calm down its just a copy pasta joke

[mahyar]

Just now, eeeee1 said:

calm down its just a copy pasta joke

im not angry just making things clear

[eeeee1]

Just now, mahyar said:

im not angry just making things clear

then appreciated

[WereCatf]

22 minutes ago, Grabinio said:

My question is there any suggestions on what I can change or what is impossible to change as a team of 2 individuals ?. For example can I create a package manger?

Just the act of creating your own package-manager can be a really big undertaking in and of itself: you'll have to figure out how to track dependencies, how to manage conflicts, track changes to files and which kinds of changes to handle in what manner -- e.g. configuration-files --, you'd most likely want to create some sort of a chain-of-trust, version-tracking, and so on and so forth.

 

A good, modern package-manager in and of itself would most likely suffice as the entire project, especially so, if you manage to make it work across multiple distros!

22 minutes ago, Grabinio said:

I really want to do this from scratch but in order to define a real clear  goal I have to know what I can or cannot modify before I even start working on it. Can someone clarify this for me or do you think that I should just drop this whole idea and look for another project?

This doesn't really make much sense, since you can modify any open-source part you like. Whether you have the skills and the time to do so, however, you'll have to answer for yourself.

 

EDIT: I thought I should possibly mentioned that I've used Gentoo for years and made my own, customized easily-installable Gentoo "distro-lite" that booted off of CD-ROM and USB; even that was a ton of work and LFS is far more work.

[Guest]

25 minutes ago, Grabinio said:

Also I will learn a lot about Linux which I personally think is important nowadays.

You would be better of playing around in Arch (Pre-Compiled) or Gentoo(Source Based) than dealing with LFS( Source Based, No pre testing, no predefine configurations, etc...) to learn how Linux works.

 

25 minutes ago, Grabinio said:

So I have no idea where is the limit as it is something you find whilst building the distro I plan on using Linux from Scratch book as a guide.

I am not entirely sure what you mean here. Building a Distro from scratch will be time consuming, not entirely difficult. A Single person can easily build a LFS Install, it's maintaining it that becomes the burden. If you turn it into a actual Distro, that burden amplifies quite drastically. This is why many Distros use other distros like ubuntu and debian as bases. Maintaining just a Distro based on another Distro however can also be quite a burden, as your still responsible for making sure everything works as expected and keeping your user base somewhat happy. These reasons are why Distro's come and go a lot, apart from the fact that so many already exist.

 

25 minutes ago, Grabinio said:

For example can I create a package manger?

You can, but your will have to make it. It will also need to pull from a repo of some sort, whether its source based or precompiled. You will also need to define where files are placed in the file-system and have a way to track the files, versions, origin, etc... You will also need to make and find a way to host this repo.

Here is the source tree of dnf(fedora) and pacman(Arch) for reference.

https://github.com/rpm-software-management/dnf

https://git.archlinux.org/pacman.git/tree/

 

25 minutes ago, Grabinio said:

The issue is that I have zero experience in Linux development

You really need to be familiar with Linux Development in General to take on something like LFS or Gentoo, let alone building a package manager.

 

25 minutes ago, Grabinio said:

I really want to do this from scratch but in order to define a real clear  goal I have to know what I can or cannot modify before I even start working on it. Can someone clarify this for me or do you think that I should just drop this whole idea and look for another project?.

You can modify anything that is open-source, you would need to figure out what you want to use. There is no set packages in Linux, just common ones.

Trust me, you really don't and really doesn't serve much purpose. I would start with something much smaller. Linux needs lots of love in the Graphical UI field, maybe find something interesting that doesn't have one and build on it.

[Grabinio]

1 hour ago, Radium_Angel said:

By and large, re-mixes of existing distros will require a small team of talented folks to work on.

Creating your own *nix and package manager from the ground up, well..I hope you are a kick-ass programmer.

You can start at www.distrowatch.com and dig around, though you *might* be interested in this specific linux

https://distrowatch.com/table.php?distribution=scientific

 

In particular, read who developed it and why.

 

I think creating a custom PM (package manager) will be beyond your time frame at University, but a remix of an existing distro with student engineering in mind might be in the realm of possibility for you.

 

Either way, you are in for a lot of work. Good luck.

That is exactly why I am asking. Now I know that putting "Creating a package manager" under specifications or goals is not possible.
I am not a kick ass programmer for sure am your average Joe . But extremely stubborn , can work harder than a peasant though.

Thank you.

@Radium_Angel

[Grabinio]

1 hour ago, WereCatf said:

Just the act of creating your own package-manager can be a really big undertaking in and of itself: you'll have to figure out how to track dependencies, how to manage conflicts, track changes to files and which kinds of changes to handle in what manner -- e.g. configuration-files --, you'd most likely want to create some sort of a chain-of-trust, version-tracking, and so on and so forth.

 

So it is not possible in my time frame. Can I just skip creating it and use APT for example?
@WereCatf

 

 

1 hour ago, WereCatf said:

 

 

A good, modern package-manager in and of itself would most likely suffice as the entire project, especially so, if you manage to make it work across multiple distros!

This doesn't really make much sense, since you can modify any open-source part you like. Whether you have the skills and the time to do so, however, you'll have to answer for yourself.

 

I apologize for not clearing what I meant by can or cannot modify. What i mean is how advanced is the code? is it possible in the time frame?.
The answer for yourself part is what making it hard. How can I know if I will be able to do it or not?. I would love to work until l hit a wall but they don't want the incremental approach.

Do you have certain ideas on what software I have the time do reproduce or modify in a certain way?.
Or what is too time consuming it is better to leave it as it is

[Kilrah]

3 minutes ago, Grabinio said:

How can I know if I will be able to do it or not?. I would love to work until l hit a wall but they don't want the incremental approach.

Start looking at it now, in preparation. See for yourself how you understand it/feel about it - only then will you be able to know if that's a viable project for you.

 

There's no miracle, we can't know your skills. And you can't jump into complete unknown and commit to a project when you don't yet even know what any of it is about...

[Grabinio]

1 hour ago, Nayr438 said:

 

I am not entirely sure what you mean here. Building a Distro from scratch will be time consuming, not entirely difficult. A Single person can easily build a LFS Install, it's maintaining it that becomes the burden. If you turn it into a actual Distro, that burden amplifies quite drastically. This is why many Distros use other distros like ubuntu and debian as bases. Maintaining just a Distro based on another Distro however can also be quite a burden, as your still responsible for making sure everything works as expected and keeping your user base somewhat happy. These reasons are why Distro's come and go a lot, apart from the fact that so many already exist.

 

What I mean from scratch is following the LFS book to a certain extent and modifying certain things along the way. Maintaining is not an issue at the moment, If I gain good experience from this I will sure build something better to maintain later.
@Nayr438

 

 

1 hour ago, Nayr438 said:

You really need to be familiar with Linux Development in General to take on something like LFS or Gentoo, let alone building a package manager.

 

Is LFS really hard to follow? are there more books or guide that you know of can help?
 

 

1 hour ago, Nayr438 said:

You can modify anything that is open-source, you would need to figure out what you want to use. There is no set packages in Linux, just common ones.

Trust me, you really don't and really doesn't serve much purpose. I would start with something much smaller. Linux needs lots of love in the Graphical UI field, maybe find something interesting that doesn't have one and build on it.

I would love to do that , the problem is that I have to specify what I want to modify before starting the project I cant do that because I do not know how long will it take to finish building the distro it self . BTW do you know if following the LFS will take too much time?


Thank you

[Grabinio]

9 minutes ago, Kilrah said:

Start looking at it now, in preparation. See for yourself how you understand it/feel about it - only then will you be able to know if that's a viable project for you.

 

There's no miracle, we can't know your skills. And you can't jump into complete unknown and commit to a project when you don't yet even know what any of it is about...

@Kilrah What you say is really reasonable.
What I do know is that I want to invest my time and effort and do it.
Not asking for any miracles, just a little bit of what people know or experienced or if any tried to do it.
What am trying to do here is turn that complete unknown into a little bit known. Then I can tell if I should give up or not.

[Sauron]

1 hour ago, Grabinio said:

My professor states that the idea in general is OK, but the end goal is not clear.

Well yeah, "A Linux distro" is extremely vague... what features would you want it to have? What tools do you plan on using to accomplish that goal? Do you plan on changing any code for anything or would you just be combining what others did (as LFS teaches you to do by the way)?

1 hour ago, Grabinio said:

I explained that basically most distros are the same, it is just different packages and looks.

Not true, but either way you should definitely have a clear idea for how your distro should be different from others.

1 hour ago, Grabinio said:

I do not think I can specify what I can modify or what I can create by myself or what I can reuse as I said I have never done this before the only why I can find this is whilst doing it. Perhaps a certain software needs a huge team to work on I cant say that I will modify it or create something similar then fail to deliver.

Then honestly... change project and do something you have at least a vague idea on how to do.

1 hour ago, Grabinio said:

can I create a package manger?

Yes, but this is a large enough task to be its own project. In fact, I find this to be a much better goal than just "making a distro". You can use LFS as a base system to develop your package manager on. I'm sure your professor would be much more interested in this specifically rather than the final Linux system you end up with.

[WereCatf]

38 minutes ago, Grabinio said:

So it is not possible in my time frame. Can I just skip creating it and use APT for example?

I have no idea how easily APT can be adapted for distros that aren't based on Debian/Ubuntu, but it'd probably require quite a lot of work.

40 minutes ago, Grabinio said:

What i mean is how advanced is the code?

Code of which part? A typical Linux-distro is comprised of thousands of packages...

41 minutes ago, Grabinio said:

How can I know if I will be able to do it or not?. I would love to work until l hit a wall but they don't want the incremental approach

At this point, I would recommend you to install Gentoo in a VM and get a feel. Gentoo could be described as LFS-on-training-wheels -- it does a lot of the work for you, but it definitely doesn't prevent you from completely borking everything up. There is literally no way of understanding the scale of all the work that goes into a distro without getting your hands dirty.

[Guest]

1 hour ago, Grabinio said:

What I mean from scratch is following the LFS book to a certain extent and modifying certain things along the way. Maintaining is not an issue at the moment, If I gain good experience from this I will sure build something better to maintain later.

The problem is, LFS won't necessarily be representative of another Distro, so I am not sure of what you mean to a certain extent. If your not familiar with Linux, you are going to need to base it off something.

 

1 hour ago, Grabinio said:

Is LFS really hard to follow? are there more books or guide that you know of can help?

Not hard to follow, but if you run into any issues along they way, you will have to figure out it on your own. Everything in LFS is built from source, so you will be compiling and configuring everything on your own. If your not familiar with how Linux Development works in general, then your just building it blindly with no idea what to do if something goes wrong and no idea how anything really works.

 

1 hour ago, Grabinio said:

I would love to do that , the problem is that I have to specify what I want to modify before starting the project I cant do that because I do not know how long will it take to finish building the distro it self . BTW do you know if following the LFS will take too much time?

How long it takes to setup LFS, depends on how familiar you are with Linux, How fast your system can compile packages, and how good you are at problem solving in the event something goes wrong. So that's a hard question to answer, you could do it in a day or it could take you couple weeks, its hard to say. The kernel alone for example on a low end system could take an entire day to compile, but only a hour on a more powerful system. Main Distros use automated build servers for this that are constantly building packages.

 

In terms on how long it will take to build a full Distro itself, that's to vague of a question with no real answer. It comes down to how familiar you are with Linux, what you are doing with it, what packages you want to use, if you decide to base it off LFS or use another distro as a base. Keep in mind most Distros just use a base such as Ubuntu, swap out a few packages, then make changes over the course of several years with a handful of people and community volunteers.

 

1 hour ago, Grabinio said:

What am trying to do here is turn that complete unknown into a little bit known. Then I can tell if I should give up or not.

Install a Distro Like Arch, Familiarize yourself with it, download the source to various packages, look through them, try modifying and building them. Once your comfortable with that then move onto something like Gentoo and familiarize yourself with it as well, then you can determine if LFS or making a Distro is something you really want to do.

 

Really what your asking for, only you can answer.

[Grabinio]

Thank you all.

This is helpful.