Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
firelighter487

how can i run terminal commands on all core's?

Recommended Posts

Posted · Original PosterOP

compiling from the AUR etc takes forever for me because it only does it on 1 core. how can i make it use all 4 core's?


DISCLAIMER: ANYTHING I SAY COULD BE WRONG. DO YOUR OWN RESEARCH!

THIS STATUS CHANGES OVER TIME. REFERENCES TO IT ARE NOT ALWAYS ACCURATE. 

PC: i5 7400 | GTX 780Ti | 2x 240GB SSD | 16GB RAM | AX860 | Manjaro

MACBOOK PRO: I7 2675QM | HD 3000 | 16GB RAM | 500GB SSD | Manjaro

KEYBOARD & MOUSE | CORSAIR K95 RGPLATINUM (MX BROWN) | CORSAIR M65 PRO RGB

Link to post
Share on other sites
Posted · Original PosterOP
11 minutes ago, M.Yurizaki said:

Would probably better hit the notification thingy: https://www.reddit.com/r/archlinux/comments/494c84/speed_up_compiling_of_aur_packages/

 

I'm assuming this is what you're talking about.

didn't work. i set it to 4 and it still only uses 1 core :( 


DISCLAIMER: ANYTHING I SAY COULD BE WRONG. DO YOUR OWN RESEARCH!

THIS STATUS CHANGES OVER TIME. REFERENCES TO IT ARE NOT ALWAYS ACCURATE. 

PC: i5 7400 | GTX 780Ti | 2x 240GB SSD | 16GB RAM | AX860 | Manjaro

MACBOOK PRO: I7 2675QM | HD 3000 | 16GB RAM | 500GB SSD | Manjaro

KEYBOARD & MOUSE | CORSAIR K95 RGPLATINUM (MX BROWN) | CORSAIR M65 PRO RGB

Link to post
Share on other sites

Depends what you're compiling.  Make man not be able to parallelize the job.   I'm a bit surprised anyone would suggest using "make -j $(nproc)" though, that is silly for end users.

Link to post
Share on other sites
Posted · Original PosterOP
1 minute ago, toor said:

Depends what you're compiling.  Make man not be able to parallelize the job.   I'm a bit surprised anyone would suggest using "make -j $(nproc)" though, that is silly for end users.

just installing stuff from the AUR... 


DISCLAIMER: ANYTHING I SAY COULD BE WRONG. DO YOUR OWN RESEARCH!

THIS STATUS CHANGES OVER TIME. REFERENCES TO IT ARE NOT ALWAYS ACCURATE. 

PC: i5 7400 | GTX 780Ti | 2x 240GB SSD | 16GB RAM | AX860 | Manjaro

MACBOOK PRO: I7 2675QM | HD 3000 | 16GB RAM | 500GB SSD | Manjaro

KEYBOARD & MOUSE | CORSAIR K95 RGPLATINUM (MX BROWN) | CORSAIR M65 PRO RGB

Link to post
Share on other sites
8 hours ago, firelighter487 said:

compiling from the AUR etc takes forever for me because it only does it on 1 core. how can i make it use all 4 core's?

Edit  /etc/makepkg.conf

look for the commented line that says

#MAKEFLAGS="-j2"

and remove the comment tag (#), substitute the 2 with the number of cores you have. You can read about this on the arch wiki as well https://wiki.archlinux.org/index.php/Makepkg#Parallel_compilation


<Make me a sandwich.> <No! Make it yourself!> <Sudo make me a sandwich.> <FINE.> What is scaling and how does it work? Asus PB287Q unboxing! Console alternatives :D  CoC F.A.Q Beginner's Guide To LTT (by iamdarkyoshi)

Sauron'stm Product Scores:

Spoiler

Just a list of my personal scores for some products, in no particular order, with brief comments. I just got the idea to do them so they aren't many for now :)

Don't take these as complete reviews or final truths - they are just my personal impressions on products I may or may not have used, summed up in a couple of sentences and a rough score. All scores take into account the unit's price and time of release, heavily so, therefore don't expect absolute performance to be reflected here.

 

-Lenovo Thinkpad X220 - [8/10]

Spoiler

A durable and reliable machine that is relatively lightweight, has all the hardware it needs to never feel sluggish and has a great IPS matte screen. Downsides are mostly due to its age, most notably the screen resolution of 1366x768 and usb 2.0 ports.

 

-Apple Macbook (2015) - [Garbage -/10]

Spoiler

From my perspective, this product has no redeeming factors given its price and the competition. It is underpowered, overpriced, impractical due to its single port and is made redundant even by Apple's own iPad pro line.

 

-OnePlus X - [7/10]

Spoiler

A good phone for the price. It does everything I (and most people) need without being sluggish and has no particularly bad flaws. The lack of recent software updates and relatively barebones feature kit (most notably the lack of 5GHz wifi, biometric sensors and backlight for the capacitive buttons) prevent it from being exceptional.

 

-Microsoft Surface Book 2 - [Garbage - -/10]

Spoiler

Overpriced and rushed, offers nothing notable compared to the competition, doesn't come with an adequate charger despite the premium price. Worse than the Macbook for not even offering the small plus sides of having macOS. Buy a Razer Blade if you want high performance in a (relatively) light package.

 

-Intel Core i7 2600/k - [9/10]

Spoiler

Quite possibly Intel's best product launch ever. It had all the bleeding edge features of the time, it came with a very significant performance improvement over its predecessor and it had a soldered heatspreader, allowing for efficient cooling and great overclocking. Even the "locked" version could be overclocked through the multiplier within (quite reasonable) limits.

 

-Apple iPad Pro - [5/10]

Spoiler

A pretty good product, sunk by its price (plus the extra cost of the physical keyboard and the pencil). Buy it if you don't mind the Apple tax and are looking for a very light office machine with an excellent digitizer. Particularly good for rich students. Bad for cheap tinkerers like myself.

 

 

Link to post
Share on other sites
7 hours ago, toor said:

I'm a bit surprised anyone would suggest using "make -j $(nproc)" though, that is silly for end users.

Since these are AUR packages it's usually safe to assume the maintainers tested them and made sure they work with -j flags. At worst it won't be any faster.


<Make me a sandwich.> <No! Make it yourself!> <Sudo make me a sandwich.> <FINE.> What is scaling and how does it work? Asus PB287Q unboxing! Console alternatives :D  CoC F.A.Q Beginner's Guide To LTT (by iamdarkyoshi)

Sauron'stm Product Scores:

Spoiler

Just a list of my personal scores for some products, in no particular order, with brief comments. I just got the idea to do them so they aren't many for now :)

Don't take these as complete reviews or final truths - they are just my personal impressions on products I may or may not have used, summed up in a couple of sentences and a rough score. All scores take into account the unit's price and time of release, heavily so, therefore don't expect absolute performance to be reflected here.

 

-Lenovo Thinkpad X220 - [8/10]

Spoiler

A durable and reliable machine that is relatively lightweight, has all the hardware it needs to never feel sluggish and has a great IPS matte screen. Downsides are mostly due to its age, most notably the screen resolution of 1366x768 and usb 2.0 ports.

 

-Apple Macbook (2015) - [Garbage -/10]

Spoiler

From my perspective, this product has no redeeming factors given its price and the competition. It is underpowered, overpriced, impractical due to its single port and is made redundant even by Apple's own iPad pro line.

 

-OnePlus X - [7/10]

Spoiler

A good phone for the price. It does everything I (and most people) need without being sluggish and has no particularly bad flaws. The lack of recent software updates and relatively barebones feature kit (most notably the lack of 5GHz wifi, biometric sensors and backlight for the capacitive buttons) prevent it from being exceptional.

 

-Microsoft Surface Book 2 - [Garbage - -/10]

Spoiler

Overpriced and rushed, offers nothing notable compared to the competition, doesn't come with an adequate charger despite the premium price. Worse than the Macbook for not even offering the small plus sides of having macOS. Buy a Razer Blade if you want high performance in a (relatively) light package.

 

-Intel Core i7 2600/k - [9/10]

Spoiler

Quite possibly Intel's best product launch ever. It had all the bleeding edge features of the time, it came with a very significant performance improvement over its predecessor and it had a soldered heatspreader, allowing for efficient cooling and great overclocking. Even the "locked" version could be overclocked through the multiplier within (quite reasonable) limits.

 

-Apple iPad Pro - [5/10]

Spoiler

A pretty good product, sunk by its price (plus the extra cost of the physical keyboard and the pencil). Buy it if you don't mind the Apple tax and are looking for a very light office machine with an excellent digitizer. Particularly good for rich students. Bad for cheap tinkerers like myself.

 

 

Link to post
Share on other sites
Posted · Original PosterOP
10 minutes ago, Sauron said:

Edit  /etc/makepkg.conf

look for the commented line that says


#MAKEFLAGS -j2

and remove the comment tag (#), substitute the 2 with the number of cores you have. You can read about this on the arch wiki as well https://wiki.archlinux.org/index.php/Makepkg#Parallel_compilation

for me the -j2 was like this: "-j2". i've edited it now with the quotes still there. is that ok? 

 

also, does it count core's or threads? in other words do i need to set it at 4 or 8 on my laptop, because on that i have 8 threads... 


DISCLAIMER: ANYTHING I SAY COULD BE WRONG. DO YOUR OWN RESEARCH!

THIS STATUS CHANGES OVER TIME. REFERENCES TO IT ARE NOT ALWAYS ACCURATE. 

PC: i5 7400 | GTX 780Ti | 2x 240GB SSD | 16GB RAM | AX860 | Manjaro

MACBOOK PRO: I7 2675QM | HD 3000 | 16GB RAM | 500GB SSD | Manjaro

KEYBOARD & MOUSE | CORSAIR K95 RGPLATINUM (MX BROWN) | CORSAIR M65 PRO RGB

Link to post
Share on other sites
2 minutes ago, firelighter487 said:

for me the -j2 was like this: "-j2". i've edited it now with the quotes still there. is that ok?

Yeah, I forgot the quotes. The wiki has the right syntax.

3 minutes ago, firelighter487 said:

also, does it count core's or threads? in other words do i need to set it at 4 or 8 on my laptop, because on that i have 8 threads... 

make will spawn as many threads as the number after the j, so yes, hyperthreading counts. You should use 8.


<Make me a sandwich.> <No! Make it yourself!> <Sudo make me a sandwich.> <FINE.> What is scaling and how does it work? Asus PB287Q unboxing! Console alternatives :D  CoC F.A.Q Beginner's Guide To LTT (by iamdarkyoshi)

Sauron'stm Product Scores:

Spoiler

Just a list of my personal scores for some products, in no particular order, with brief comments. I just got the idea to do them so they aren't many for now :)

Don't take these as complete reviews or final truths - they are just my personal impressions on products I may or may not have used, summed up in a couple of sentences and a rough score. All scores take into account the unit's price and time of release, heavily so, therefore don't expect absolute performance to be reflected here.

 

-Lenovo Thinkpad X220 - [8/10]

Spoiler

A durable and reliable machine that is relatively lightweight, has all the hardware it needs to never feel sluggish and has a great IPS matte screen. Downsides are mostly due to its age, most notably the screen resolution of 1366x768 and usb 2.0 ports.

 

-Apple Macbook (2015) - [Garbage -/10]

Spoiler

From my perspective, this product has no redeeming factors given its price and the competition. It is underpowered, overpriced, impractical due to its single port and is made redundant even by Apple's own iPad pro line.

 

-OnePlus X - [7/10]

Spoiler

A good phone for the price. It does everything I (and most people) need without being sluggish and has no particularly bad flaws. The lack of recent software updates and relatively barebones feature kit (most notably the lack of 5GHz wifi, biometric sensors and backlight for the capacitive buttons) prevent it from being exceptional.

 

-Microsoft Surface Book 2 - [Garbage - -/10]

Spoiler

Overpriced and rushed, offers nothing notable compared to the competition, doesn't come with an adequate charger despite the premium price. Worse than the Macbook for not even offering the small plus sides of having macOS. Buy a Razer Blade if you want high performance in a (relatively) light package.

 

-Intel Core i7 2600/k - [9/10]

Spoiler

Quite possibly Intel's best product launch ever. It had all the bleeding edge features of the time, it came with a very significant performance improvement over its predecessor and it had a soldered heatspreader, allowing for efficient cooling and great overclocking. Even the "locked" version could be overclocked through the multiplier within (quite reasonable) limits.

 

-Apple iPad Pro - [5/10]

Spoiler

A pretty good product, sunk by its price (plus the extra cost of the physical keyboard and the pencil). Buy it if you don't mind the Apple tax and are looking for a very light office machine with an excellent digitizer. Particularly good for rich students. Bad for cheap tinkerers like myself.

 

 

Link to post
Share on other sites

It should work with or without quotes. So either:

 

MAKEFLAGS="-j6"

or

MAKEFLAGS=-j6

 

Should be fine (quotes are necessary if you have spaces, say adding several MAKEFLAGS; and with that rationale the quotes are more clean or correct, because it will be easier to add flags later if needed).

 

Some programs might compile / parallelize worse than others, some might have some weird (non-standard) way to call make or compiling the source and drop the MAKEFLAGS variable set in /etc/makepkg.conf. In that case you could try to fix the PKGBUILD.

 

IMO it is a bad habit to set MAKEFLAGS in the PKGBUILD (unless there are specific reasons to do so); Usually they are system specific and should be set on the host system.

Link to post
Share on other sites
1 hour ago, Wild Penquin said:

IMO it is a bad habit to set MAKEFLAGS in the PKGBUILD (unless there are specific reasons to do so); Usually they are system specific and should be set on the host system.

 

That was more where I was heading.  Just because you have 32 cores doesn't mean you get the best performance out of pegging them all at once.   You'll find docs saying to rung make -j (no limit), something that is quite assinine since you can easily cause essentially a forkbomb.     Depending on the build system you can limit based on other metrics.   Example is make has -n for load.

Link to post
Share on other sites

 

On 10/11/2018 at 2:14 PM, Wild Penquin said:

It should work with or without quotes. So either:

 


MAKEFLAGS="-j6"

or


MAKEFLAGS=-j6

 

Should be fine (quotes are necessary if you have spaces, say adding several MAKEFLAGS; and with that rationale the quotes are more clean or correct, because it will be easier to add flags later if needed).

 

Some programs might compile / parallelize worse than others, some might have some weird (non-standard) way to call make or compiling the source and drop the MAKEFLAGS variable set in /etc/makepkg.conf. In that case you could try to fix the PKGBUILD.

 

IMO it is a bad habit to set MAKEFLAGS in the PKGBUILD (unless there are specific reasons to do so); Usually they are system specific and should be set on the host system.

This is correct, follow that directions, however note that not all AUR packages use the same compiler, this is for GCC, for example clang works even without those build flags, other programs may compile on one core just for compatibility

Anyway I would suggest to use the quotes. By default they should put something like MAKEFLAGS="-j$(nproc)" which automatically detects the number of threads but whatever...
Also you can speed up the package compression, in the same file search for 

COMPRESSXZ=(xz -c -z - --threads=0) 

And edit the number of threads your CPU has

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×