Need a solid template for a C/C++ build system.

[Haswellx86]

What I find a lot of C/C++ projects lacking is a good build system. I wish we had a standard build system with standard profiles that everybody used. Instead, we have conventions such as a bin directory or build directory but it is upon the maintainer to follow the conventions.

 

I am not sure if my idea is not that proper or impractical because I am probably the only one looking for something like this because if not, there should have been such a standard by now.

 

I am looking for a build system template that supports- 

1. multiple operating systems (Win32, UNIX and OS X),
2. multiple compiler toolchains or at least compiler architectures (this step is already where most long projects fail; LLVM adoption is slow) such as GCC, MSVC, Clang, ICX, etc (these compilers really should have a standard user front-end for compatibility),
3. good and standard directory structure (something like build/release/distributable/x86_64/program),
4. standard profiles for building the project. A lot of projects don't have that many profiles which configure the compiler with different compiler options and flags. Some only do till -O2 while -O3 does actually not break code nowadays and is better. Some do -g (debug) for a release build. If I am doing a release build, I do not care a single bit about debugging. I have compiled a few programs by forcefully modifying the Makefile to remove debugging when I don't need it. I have rarely seen anyone strip the binary. And many of them don't even specify -march and -mtune options for the compiler to use better instruction sets. Only low level or performance critical programs use these options but in my opinion all programs should be compiled with better architectural support.

The most apparent build system is CMake. I was actually putting some time creating such a template but if I want to it the most proper way such as using many GCC compiler options efficiently and for formality, there will be someone who knows much better than me. I don't think anyone can make use of most of the options GCC provides.

[wasab]

Standarizing build tools is kinda like standardizing operating systems and demanding macos, windows, Linux and what have you all function the same. Of course you can't do that.

 

Make is part of the gnu toolchain so you can imagine the issues if you want to use it on windows and similarly for Microsoft msbuild if you want to use it on Linux. Both do support the other platforms but they are nowhere the default go to options and generally more tedious to use than the "official" tools for their respective os.  

 

CMake is basically a build tool for the build tools. It will generate the proper build files and configuration for your particular operating system and environment. This means, of course, you need different flags and configuration to handle the differences of each of the target platforms, compilers, and native shared libraries/dependencies for the specific platform you wish to cross-compile for. This is on top of avoiding all native APIs and headers within the source code itself, like windows.h or unistd.h or at least lots of preprocessor directives to selectively include/omit APIs based on the target operating system. 

 

If you don't like that, pick a platform-agnostic language like Java. This thing is literally write once, compile once, and run everywhere. After you built, compiled, and packaged your Java code into a jar file, you can literally copy and paste it into a thumb drive, move it to another operating system with a java virtual machine, and it will run completely out of the box and behave exactly the same. This means standardizing build toolchains for java applications is completely irrelevant and a non-issue (you don't need to compile for each operating system and machine).

 

**There are edge cases and caveats ,but I am generalizing here. 

[Haswellx86]

9 hours ago, wasab said:

Standarizing build tools is kinda like standardizing operating systems and demanding macos, windows, Linux and what have you all function the same. Of course you can't do that.

That would like saying stdio.h doesn't work on Windows. Compatibility and re-implementation can support this.

 

9 hours ago, wasab said:

Make is part of the gnu toolchain so you can imagine the issues if you want to use it on windows and similarly for Microsoft msbuild if you want to use it on Linux. Both do support the other platforms but they are nowhere the default go to options and generally more tedious to use than the "official" tools for their respective os. 

I know and that is why I was talking about CMake.

 

9 hours ago, wasab said:

This means, of course, you need different flags and configuration to handle the differences of each of the target platforms, compilers, and native shared libraries/dependencies for the specific platform you wish to cross-compile for. This is on top of avoiding all native APIs and headers within the source code itself, like windows.h or unistd.h or at least lots of preprocessor directives to selectively include/omit APIs based on the target operating system. 

Yes I am aware. That is why I wonder why nobody has written a proper build system template that takes care of all of these. Looks like everyone just kind of have their own custom build system with whatever knowledge of compiler options they know of. I must build my own CMake template then.

 

9 hours ago, wasab said:

If you don't like that, pick a platform-agnostic language like Java. This thing is literally write once, compile once, and run everywhere. After you built, compiled, and packaged your Java code into a jar file, you can literally copy and paste it into a thumb drive, move it to another operating system with a java virtual machine, and it will run completely out of the box and behave exactly the same. This means standardizing build toolchains for java applications is completely irrelevant and a non-issue (you don't need to cross-compile for each operating system and machine).

Right. I must do OS development in Java. Damn it, I have been doing it wrong the whole time!

[wasab]

10 hours ago, Haswellx86 said:

That would like saying stdio.h doesn't work on Windows. Compatibility and re-implementation can support this.

stdio.h is part of libc, so of course it would be compatible everywhere. it is a requirement for a programming language to be called C but you are grossly trivializing the difficulty in writing cross-platform code. Unless you want to write a very basic command-line utility, you need more than the standard library. I will give you a basic example. Have you ever written a cli shell? How it works is basically to take in the user input from stdin, spin up a child process, then within that child process, load up an entirely new process image that corresponds to the command, assuming that the command is an external terminal application. 

 

straightforward right? But how do you spin up a child process? well, this is how you do it in Unix-based operating system

https://www.man7.org/linux/man-pages/man2/fork.2.html

 

how about on windows? it is this

https://learn.microsoft.com/en-us/windows/win32/procthread/creating-processes

 

These two have NOTHING in common. if you want to write a shared common code base that compiles to both operating system, you would mostly likely need to either import cross-platform, third party wrapper that does this, or write one up yourself if you are hardcore which mostly involves lots of code that essentially do the same thing but for each specific platform you want to compile to. This begs the question, why would you want the headache of doing this? Writing bunch of code that does the same thing?

 

Another basic example is lets say you need to import a library to make your application work. i did this for a c project i wrote. the library is libcrypto, which is basically the open-source SSL library for encryption. Also simple and straightforward right? 

 

well, it turns out this library does not exist on windows

https://stackoverflow.com/questions/19321982/libcrypto-equivalent-missing-on-windows

 

you have to import another equivalent and god forbid if its interfaces are different from whats found in libcrypto because if it is, guess what? you are doing the same thing as the previous example. write codes that do the exact same thing but for different platforms. 

 

Just for these differences, such as different libraries, you probably wont have a single cmake file template that fits for all. Depending on the operating system, you will need to tell cmake to link different libraries, for example. 

 

You can say yeah, let's just rewrite and reimplement all of these so you get fork on windows and createprocess on Linux but why are you reinventing the wheels? Why would you want the headaches of maintaining two separate APIs that do the same thing? Okay, so how about we just completely deprecate one and favors another?

 

Oh boy, you must be joking then because this breaks ALL backward compatibility for existing applications. Developers of these app would be screaming and rioting so you see? You are grossly trivializing the task and difficulty.

 

POSIX is probably the closest we have to this and even then a lot is left out. A standardized GUI library is one example. Good luck writing a cross platform GUI app. GUI is inherently and extremely OS specific, even more so if you want the UI to look and feel native. You will need something like QT framework, or better yet do away with it altogether and just use the web (html/css/JavaScript) and browser as your UX rather than standarizing it into POSIX or whatever. 

 

10 hours ago, Haswellx86 said:

I know and that is why I was talking about CMake.

Okay, i already tell you why you cant have a one-size-fits-all all option.

 

10 hours ago, Haswellx86 said:

Yes I am aware. That is why I wonder why nobody has written a proper build system template that takes care of all of these. Looks like everyone just kind of have their own custom build system with whatever knowledge of compiler options they know of. I must build my own CMake template then.

right, you cant and I explained above. it is not just the build tools. There are also differences in compiler flags, the linker flags, the dependencies ect.

 

10 hours ago, Haswellx86 said:

Right. I must do OS development in Java. Damn it, I have been doing it wrong the whole time!

if you are building and compiling an OS, why would you even need to worry about cross-platform and cross-compiling to begin with? Just use whatever build tools and configurations their documentation says. 

[Haswellx86]

2 hours ago, wasab said:

I will give you a basic example. Have you ever written a cli shell? How it works is basically to take in the user input from stdin, spin up a child process, then within that child process, load up an entirely new process image that corresponds to the command, assuming that the command is an external terminal application. 

 

straightforward right? But how do you spin up a child process? well, this is how you do it in Unix-based operating system

https://www.man7.org/linux/man-pages/man2/fork.2.html

 

how about on windows? it is this

https://learn.microsoft.com/en-us/windows/win32/procthread/creating-processes

I am aware of how to create a new process in C on both the platforms. That is still a relatively easy example and I am sure wrappers exist to dispatch their calls to the operating system equivalent.

 

2 hours ago, wasab said:

These two have NOTHING in common. if you want to write a shared common code base that compiles to both operating system, you would mostly likely need to either import cross-platform, third party wrapper that does this, or write one up yourself if you are hardcore which mostly involves lots of code that essentially do the same thing but for each specific platform you want to compile to. This begs the question, why would you want the headache of doing this? Writing bunch of code that does the same thing?

Yeah so somebody has to write wrappers to make other people's life easier, no? 

 

I am not saying that I am willing to write a wrapper but also you are talking deep into inter-platform coding where I was talking about just a CMake template. The template just creates a directory structure, and has many compilation profiles which call the compiler with their compatible options optimized for each profile. That's the template. The rest such as one library not available on the other platform will be handled by the programmer. They need to fill in the template after all.

 

Also you seem to have misunderstood the meaning of "supports different operating systems". I am talking about the build system (which is CMake which is cross platform) and not if a programmer needs to support multiple platforms. I need people on Linux or Windows to be able to run the configured build system and not necessarily need to compile on each other. You could even suggest me different build systems for the platforms but all I am looking for is a pre-configured template.

 

2 hours ago, wasab said:

Just for these differences, such as different libraries, you probably wont have a single cmake file template that fits for all. Depending on the operating system, you will need to tell cmake to link different libraries, for example.

Yes, of course the programmer has to do the work to link different libraries according to the operating system. That doesn't come in the *template*. How would the template know what libraries you are going to need anyway?

 

2 hours ago, wasab said:

You can say yeah, let's just rewrite and reimplement all of these so you get fork on windows and createprocess on Linux but why are you reinventing the wheels? Why would you want the headaches of maintaining two separate APIs that do the same thing? Okay, so how about we just completely deprecate one and favors another?

Yeah you totally misunderstood what I asked for as I explained above.

 

At least reinventing the wheels is good if it improves people's lives. Imagine we re-implement DirectX for Linux and instead of translating it to OpenGL or Vulkan it's a true re-implementation. That would be awesome but yeah I know, the development and maintenance will cost a lot so it is not worth it.

 

2 hours ago, wasab said:

if you are building and compiling an OS, why would you even need to worry about cross-platform and cross-compiling to begin with? Just use whatever build tools and configurations their documentation says. 

I am not gonna use that template for compiling an OS created and maintained by someone else with their own build system. I just said OS as an example related to C in that I need to program in C. Java is a completely different thing.

[wasab]

4 hours ago, Haswellx86 said:

I am aware of how to create a new process in C on both the platforms. That is still a relatively easy example and I am sure wrappers exist to dispatch their calls to the operating system equivalent.

 

Yeah so somebody has to write wrappers to make other people's life easier, no? 

doesnt exist and few bother, at least for ones that are truly portable and require no rewriting. you know how Microsoft implements POSIX apis on windows? they use the WSL(windows subsystem for linux) rather than writing wrappers.

 

4 hours ago, Haswellx86 said:

I am not saying that I am willing to write a wrapper but also you are talking deep into inter-platform coding where I was talking about just a CMake template. The template just creates a directory structure, and has many compilation profiles which call the compiler with their compatible options optimized for each profile. That's the template. The rest such as one library not available on the other platform will be handled by the programmer. They need to fill in the template after all.

if that is all you want, why wouldn't you have a standard template you could just copy and paste? There are many existing online. 

[Haswellx86]

13 hours ago, wasab said:

Microsoft implements POSIX apis on windows?

Which POSIX APIs are you talking about?

 

13 hours ago, wasab said:

hey use the WSL(windows subsystem for linux) rather than writing wrappers.

WSL is like an actual integrated thing in Windows? I thought it is just a thing used to run a Linux a distro on top of Windows. What else is it used for?

 

13 hours ago, wasab said:

if that is all you want, why wouldn't you have a standard template you could just copy and paste? There are many existing online.

That is literally what I was asking for. I can't find any good one that suits my needs. I don't know why nothing like this exists or else it would have been very popular. Let it be, I am going to try to write my own template.

[wasab]

10 hours ago, Haswellx86 said:

Which POSIX APIs are you talking about?

All of them

10 hours ago, Haswellx86 said:

WSL is like an actual integrated thing in Windows? I thought it is just a thing used to run a Linux a distro on top of Windows. What else is it used for?

It is what microsoft relies on to enable POSIX compliant environment. It is a compatibility layer if you want the short version. Cygwin is another POSIX compliant compatibility layer for windows but at the source code level; this means applications need to be recompiled, possibly also rewrite as well, in order to run, so it is a step behind wsl. 

 

 

10 hours ago, Haswellx86 said:

That is literally what I was asking for. I can't find any good one that suits my needs. I don't know why nothing like this exists or else it would have been very popular. Let it be, I am going to try to write my own template.

https://cmake.org/cmake/help/book/mastering-cmake/chapter/Cross Compiling With CMake.html

 

this one above is from their official documentation. It is about cross compiling, assuming that is what you are looking for. You can also just compile for the target platform, on the target platform instead. 

 

To be blunt, java is still the king for cross platform compatibility. The best that other programming language can do is write once, compile everywhere and that is very difficult to do on its own already. Java is literally write once AND compile once, then run everywhere. It is by far a lot more isolated from the underlying OS compared to other options out there, even compare to interpreted language like python and node. All of these low level abstraction allow most java applications to be very portable. I know you don't write java but consider what tools are at your disposal and pick the best one for your needs is always a good wisdom.