What is the difference between a library and an API?

[Gat Pelsinger]

In C, the standard input/output is a library (stdio). If I wanted to inline the assembly to make direct syscalls, I can. But I could only do that so far in Linux. On Windows, because ChatGPT is sometimes so dumb, I couldn't get any runnable program. Is it possible to directly inline the instructions to call the Windows API?

 

But my real question is, is that if stdio is a library, then why is windows.h an API? What really differentiates between a library and an API? And can't Windows directly take syscalls instead of relying on the API (wait, that still means direct syscalls, right)?

[Slottr]

https://learn.microsoft.com/en-us/windows/win32/

 

stdio is a library because its a localized functional system with no external integrations. When you call the windows.h header you're calling the windows library to interact with it internally. The API call is a wrapper for whatever the API contains - generally more libraries.

 

If you look at windows.h you'll see specifically what you're touching.

https://learn.microsoft.com/en-us/windows/win32/api/winbase/

 

You may also want to brush up on what header files actually do- and how they interact with overarching programs.

[wasab]

API: application programming interface. It is basically an outside code that got abstracted away and meant to interface with your own code. This means a library is an API since it is code written by either others or yourself which hides away inner details and only provides an entry point aka an interface, for the rest of your code to call on and interface with. this also means restful endpoints that you do http requests(GET, POST, PATCH ect) are apis as well, (called web apis) since you are invoking and interfacing with some handler/functions in the server applications to perform some tasks from your client application. The idea is that api inner workings are abstracted away and only expose an entry point for your code to interface with. For the library, this entry point is whatever the function signature is, for rest endpoints, it is the HTTP protocol and whatever parameters/payload that rest endpoints accept. Operating system calls are very much like an api if you think about it. you move some xyz input into registers, including the interrupt code for a particular service you would like to invoke, and then you do a syscall with it. 

 

library: typically packages, files, and binaries with lots of APIs. the c stdlib is referred to as libc. on Linux operating system, it is a compiled library in which many native applications and binaries are linked dynamically at run time. This is why if you delete that, you will bork your systems.

 

libc is not exclusive to C btw. many programming languages as well as the software coded with non-C language might linked to them. 

e.g. this is how you dynamically link to a c library in Java

Calling C From Java Is Easy – The Mindful Programmer (jonisalonen.com)

 

this is how you do it in Python

1. Extending Python with C or C++ — Python 3.12.2 documentation

Edit: this is the correct link

 

Edit: windows syscalls are not meant to be used directly. microsoft expects developers to use whatever system libraries and utilities they provide as a wrapper for them. You can still try but they are not publicly documented and I am pretty sure only their internal developers ever work on them.

Assembly Syscalls in 64-bit Windows - Stack Overflow

Quote

The only way to call Windows operating system functions in a way that it is guaranteed that your program is still working after the next Windows update is to call the functions in the .dll files - just the same way you would do this in a C program.

so yeah, you should probably just linked to microsoft dlls instead of reinventing the wheels. For Windows, this is the Kernel32.dll which is basically where the win32 apis come from. Note, win32 is not part of the libc. completely separate things. although on linux, the libc is analogous to kernel32.dll on windows since it is usually the lowest level codes that interface with the kernel directly and is pretty much system-wide unlike what's found on windows(windows libc is compiler specific). This makes me wonder.... what will happen if you delete kernel32.dll on windows? will it bork the os? 

[Eigenvektor]

Library: code provided by someone else. API: the interface between your code and their code.

 

You can consider the sum of all signatures of the functions provided by a library its interface. You don't need to care how stuff is implemented, you only need to care which functions it has, what arguments those take and what is returned. You don't need to care about implementation changes as long as the interface itself remains stable. While that is a given, you can update to any future version of the library without having to change your own implementation.

 

A header file declares functions (or rather their signatures), so as a whole it is an interface. The code behind those signatures is the actual library. In other words, every library has an interface.

 

~edit: But not every interface needs to belong to a library. For example the server I work on has a REST API - a web interface you can use to interact with the service/code running on our company servers. The same principle applies: We can update our code as much as we want, as long as its interface remains stable. We can add new functionality, we can fix bug, and we can deprecate stuff to (hopefully) get rid of or replace functions that are no longer useful/are broken.

[Gat Pelsinger]

@wasab

 

Yeah but then why Windows has such a problem where it cannot take syscalls directly (only technically it can), and have to rely on the DLLs? And so, if I have to call functions for syscalls, is there a performance impact (sorry I had to bring this ).

 

Edit - Actually I have another question. If I am using POSIX commands such as libc, will they eventually get translated to Windows API calls under the hood when running the code on Windows? 

[wasab]

2 hours ago, Gat Pelsinger said:

 

Yeah but then why Windows has such a problem where it cannot take syscalls directly 

One: so your code still compiles and runs in a previous version and in a future version. Do you want to rewrite your software every time whenever Windows introduces breaking changes? 

 

Two: don't reinvent the wheels. Official apis are battle tested while your own code will likely turn out to be a buggy mess. 

 

2 hours ago, Gat Pelsinger said:

 

 is there a performance impact (sorry I had to bring this ).

No. Compiler probably writes better assembly for syscall than you could. Seriously, don't go reinvent the wheels. System libraries are developed over the decades by many experienced programmers and these guys probably write better performant and optimized code than you ever could from using few hours of chatgpt. 

[Gat Pelsinger]

@wasab

3 hours ago, Gat Pelsinger said:

Actually I have another question. If I am using POSIX commands such as libc, will they eventually get translated to Windows API calls under the hood when running the code on Windows? 

 

[Gat Pelsinger]

1 hour ago, wasab said:

Seriously, don't go reinvent the wheels. System libraries are developed over the decades by many experienced programmers and these guys probably write better performant and optimized code than you ever could from using few hours of chatgpt.

Yes, I am totally aware of it. Even if I really wanted to battle the C standard, it is something I should do after I have fully mastered the language or basically became a professor. I just ask these kind of questions because I have a doubt. Here, the doubt is that are the syscalls slower because they are being called by functions in Windows? Just yes or no. I am not trying to develop a better language or OS.

[wanderingfool2]

Okay, I'm going to add a bit more confusion here as well.

 

Windows has .lib (static library files)

 

Anyways, I think the answers above have done a good job at explaining...but I'll give an analogy.

 

Library, imagine a library in real life with tons of books all categorized with the dewey decimal system.

Now imagine the API is essentially like a list of books in a particular library and their corresponding Dewey Decimal numbers.

 

To give a practical example, I am looking to read the book Dune.

I can search up the name Dune; and find the library it is in and where to find the book (this is akin to API)

The library has the book and thus I can now find it and read it (this is equivalent to the library, the actual physical books).

 

 

 

Now onto Windows stuff.  Windows has DLL and LIB files...you will commonly see DLL files, they are short for Dynamic Linked Libraries.  They are loaded when the program essentially loads them [you can even effectively replace a currently loaded DLL with another one...and since the API has essentially the information of how to find the function you don't have to worry].  This can be utilized like in emulators where plugins are released with dll files, the dll files essentially follow the API guidelines, so you can have gamepad.dll and gamepad_max.dll which both handle gamepads...but in this case gamepad_max lets say has turbo buttons implemented (this is a perfect example of using a dll).

 

Another way DLL's are used is when you essentially have multiple programs bundled together that all share the same resource, so you don't have to keep compiling with a whole library in the exe itself...or if lets say you know this piece of code will be modified a lot with frequent updates...by putting it in a dll it means it's just easier to replace the single dll and you don't have to worry about issuing updates for like 10 different exe files.

 

 

With Windows you also have LIB files, they are similar to DLL's except that when you compile your program they effectively load all the code into your executable...LIB's over DLL's should in theory be a bit faster in that instead of just lookups the compiler can hardcode locations...the issue is that you don't have the flexibility like above.

 

 

 

To answer your question above more directly, system calls are effectively API's [and some system stuff is just undocumented as well]

 

Now why you can't directly inline some of those, compiler stuff...the fact that there is lib files at play (it's already compiled, so it can't necessarily just inline those things..but I could be wrong here I never looked at the inner workings of lib files before).  I might be wrong, but I always associated LIB files as a way to release working code without actually release the source.

[wasab]

10 hours ago, Gat Pelsinger said:

Yes, I am totally aware of it. Even if I really wanted to battle the C standard, it is something I should do after I have fully mastered the language or basically became a professor. I just ask these kind of questions because I have a doubt. Here, the doubt is that are the syscalls slower because they are being called by functions in Windows? Just yes or no. I am not trying to develop a better language or OS.

No, syscalls are meant to be invoked by another piece of software and called by userland code. I don't think you are asking the right question. Functions aka a subroutine is itself just an abstraction of jump instructions to a particular instruction address. If you ever program in assembly before, you would know underhood, that a "function invocation" is nothing more than just backing up register values to the program stack, and incrementing the stack pointer, before doing a jump instruction. After the topmost subroutine finishes, it just jumps back to the previous instruction address where it jumped from, restores the register values, decrements the stack pointer and go about doing the rest of the stuff before itself gets popped off the call stack and jumps back the previous caller instruction address if any. Doing fewer function calls only means less jumping and less read/write to the stacks.

 

If you are curious, try some assembly programming to see what it is actually doing under the hood. you will quickly understand what you are asking here about performance is nonsensical. It is like asking if taking the time to start the engine on a car will cause it to drive slower. Car drives the same after the engine starts. You can probably drive the car down the road quicker if the engine starts itself but how is that useful? And more importantly why? The question doesn't make too much sense. How would you answer that? 

 

i mean to have the best performance, everyone ought to be programming in machine code but we dont. technology has developed far enough we dont need to do this just like we dont need to grow our own grain, feed it to a baby cow, raise it to maturity, then milk it whenever we want to eat cheese. instead we buy it from the supermarket from people who did all of these. do you ever ask about what grass the cows eat, what breed the cows are, whether the cheese is made of milk from Canadia cows or European cows? for most people no, why would you care? i mean you can have total control of the quality if you do everything yourself but can you do a better job? wont it be a huge waste of your time?

 

i hope i am making some sense with these far-fetched analogies. I just want to drive home the point that you should stop caring about such tiny details and rather focus on the bigger pictures. things like these are abstracted away by higher-level language for a reason so don't worry about it. 

 

Edit: I think the question you wish to ask is if these wrapper functions have any added overhead. The answer is largely no but it depends. On POSIX, many wrappers for syscalls like open, read, write ect are very thin wrappers that only move inputs to the right registers, set the right syscall call code before doing the syscall so zero overhead. On windows, idk. It might be the same or Microsoft might be doing many extra things and keep calling lower and lower library wrappers that add more extra things before finally reaching to a syscall. A lot of it is undocumented so I can't tell you. 

 

Edit2: 

Actually after some digging, the latter is the case. It just goes down to lower and lower level library until it reaches a syscall. 

https://github.com/VirtualAlllocEx/DEFCON-31-Syscalls-Workshop/wiki/04:-Chapter-2-|-Windows-OS-System-Calls

 

Still, any overhead is probably negligible since many of these seem to be just forwarding calls down rather than adding anything. 

[wanderingfool2]

4 hours ago, wasab said:

mean to have the best performance, everyone ought to be programming in machine code but we dont.

Actually even with that said, humans can no longer realistically write programs in assembly that are quicker than writing in some other languages...simply because the compiler is capable of making decisions and optimizations which humans can't practically do (even though it does end up as assembly).

 

Even some of the things you look at and say, "this must run slower" actually runs faster because the compiler has taken into account things like caches or even branch prediction optimizations.  So I'd argue, except for very specific functions things like c could be faster (or if you are an elite level assembly programmer, then you probably could...but those guys are also the ones who make those compilers)