[NASM] [Windows/Linux] [x32/x16] Getting started learning assembly

riobear1 · April 30, 2020

Hi, I've not really posted for a long time here and was bored so I thought I'd release some information for getting started with assembly, this tutorial is mainly targeted at windows; however, I will include some information for Linux platforms. I'm not particularly experienced and am able to make a few different programs but am by no means proficient at assembly. You can find some resources at the bottom of the thread. If I make any mistakes or you have anything to add, let me know so that I can update the thread

Setup
To begin with, you'll want to install NASM (by downloading it from their website) or some sort of assembler; however, I'll be using NASM because that's what I've been getting used to. Other assemblers include FASM and MASM. Remember that different assemblers, processors and operating systems will interpret assembly language differently. Specifically, I will be using Windows 10 and programming in x32 bit as I'm still trying to learn x64 bit and there are a lot more registers so it is more complex.

You'll also find that you want a linker, once again there are many linkers; however, I'll be using GoLink which you can get by searching up GoLink on your search engine.The job of the linker is to give the executable a reference to whatever dynamic libraries you may use. On Linux, an example of a linker would be GCC/G++. Proceed to add these to the path (this will vary by OS), either user or system will do but you will find it helpful if you add it to system.

Programs

Now that you've got all the required software installed, you can setup your programs directory and create the assembly file, I'd recommend using notepad++ as it has syntax highlighting for assembly language unlike many other coding environments. Another option would be something like VSCode with the extension for the syntax highlighting.

Assembly language ignores white-space (except for newlines) so remember that it'll be easier to read if you try to spread the code out into readable sections.
Now, ";" is used to create comments within code, so I'll attach a commented source code at the end for a simple hello world program, but hopefully this tutorial will help you to do that anyway.

To define the entry point for the program you will use global %Entrypoint% Where %EntryPoint% is replaced with a name, usually WinMain / Main for x32 bit programs. When using GoLink, you must use WinMain as the normal entry point as it looks for that entry point by default.

Below that you will want to import any external libraries or function through extern %FunctionName% with the function name being replaced by a WinAPI function / Function from a library.

In windows, functions are used by importing dlls whereas in linux you can also use syscodes (windows keeps them internally to prevent malicious usage), these can be found at Linux Syscodes. It is possible to find the windows syscodes; however, they vary between builds and it is generally not a good idea to use them. Instead refer to the WinAPI Docs and use the required dll which is listed and the function you want to use. Remember that you use call to call the function and push parameters in reverse notation to add them to the stack in the correct order for the function.

section .data is used to define variables/constants using the format of %VariableName% %TypeInstruction% %Value/Operand% e.g. msg db "Hello" ;DB stands for declare byte
section .bss is a section used to reserve bytes in values.
section .text is where the proper coding begins.

You can define functions using the format, %FunctionName%: and then the code under that, remember to define the entry point's function.

Then you execute your assembly instructions in the order of %Instruction% %Parameters%.

Remember, functions that return a value return it to the accumulator and to store the value you will probably need/want to move it into one of the registers.

There are different types of registers, the ones listed below are listed in order of x32 bit and x16 bit as colour coded, I skipped x8 bit and x64 because it's complicated and this is supposed to be a basic guide for people who are just starting or interested in dabbling in assembly.
The usage of each register can vary depending on the compiler, etc. but if you're following the same sort of thing as me, you probably won't see to much of a problem.

Spoiler

Code Operations

A full list of instructions can be found here -> NASM Docs; however, some example operations include:

Spoiler

push %value% - pushes value onto the stack
pop %register% - pops things off the top of the stack and pushes them to a register (As BreakShoot rightly pointed out, thanks again)
call %function - calls a function
mov %destination%, %register% - moves register data/value to destination register
sub %value%, %value% - Subtracts final value from first value
add %value%, %value% - Adds final value to first value
jmp %address% - Jumps to address, function etc.
cmp %value%, %value% - Compares two values by subtracting
je %function% - Jumps to function or address etc. if cmp was equal to 0 or the values are equal
jne %function% - Jumps if not equal to 0, if not equal
jg %function% - Jumps if greater than
jge %function% - Jumps if greater than or equal to
jl %function% - Jumps if less than
jle %function% - Jumps if less than or equal to

There are also operation conditions which are added as suffixes to an instruction, these include:

Spoiler

s - sign

z - zero

c - carry

o - overflow

Example Program

;A simple print "Hello, World!" program, this only works on windows x32bit/x64bit
;Compiled using nasm -fwin32 print.asm -o print.obj
;Linked using golink /console/entry WinMain print.obj kernel32.dll
;You can clearly see the structure of compiling and linking but I'll go into detail after

global WinMain ;EntryPoint
extern GetStdHandle, WriteConsoleA ;External imports/functions

STD_OUTPUT_HANDLE   equ -11 ;Assigned value
NULL                equ 0 ;Assigned value

section .data ;Variable definitions
msg                 db "Hello World!", 13, 10, 0 ;Message definition in appropriate format
msg.len             equ $ - msg ;Buffer parameter definition

section .bss ;Reserved stuff
dummy               resd 1 ;Reserves 4 bytes undefined space

section .text ;Actual code
WinMain: ;EntryPoint
   push    STD_OUTPUT_HANDLE ;Push parameter 1
   call    GetStdHandle ;Call GetStdHandle function
   push    NULL ;Pushes null 2 stack, 5th param
   push    dummy ;Pushes the 4 reserved bytes to stack, 4th param
   push    msg.len ;Pushes the buffer, 3rd
   push    msg ;Pushes the message, 2nd
   push    eax ;Pushes the STDOut result, 1st
   call    WriteConsoleA ;Runs the function using the 5 parameters required
   jmp    $ ;Maintains console by infinitely looping

Compiling is done with NASM by doing nasm -f *format* -o *output*
Linking is done with GoLink by doing golink /console /entry *entrypoint* *object* *dlls*

Thanks for reading through my tutorial and information on learning assembly language, I hope that you enjoyed this and have learnt a lot from it, let me know how I can improve it or if I made any mistakes, feel free to add info below and I'll update the thread!

I'll also add some useful resource websites that helped me get started:

https://nasm.us/

http://www.godevtool.com/

https://cs.lmu.edu/

https://docs.microsoft.com/en-us/windows/win32/api/

https://syscalls.kernelgrok.com/