cpu Simple Python benchmark

[Fire Yoshi]

Hey everyone,

While bored at work I made a very simple Python 3 script that requires nothing but the base libraries to benchmark single-threaded CPU performance.

It approximates π using the Gregory Series :

π = 4 - 4/3 + 4/5 - 4/7 + 4/9 ...

It runs for a set time (1 second by default) and gives the result afterwards, which is the last divider used in the series (for example, if the last operation was + 4/500, it would give 500 points).

Here it is in text, I have also attached it as a file. Feel free to post your results for your CPU (it is not very consistent, so you can run it multiple times and approximate the end result) 

 

My tests :

Intel Core i5-4590T (computer at work) : ~ 4.85M points

Intel Atom Z3560 (Asus Zenfone 2, on QPython 3) : ~ 0.226M points

from time import time
RUNTIME = 1
running = "Y"
while (running.lower()!="n"):
	start_time = time()
	divider = 1
	pi = 0
	add = True
	print("Running Pithon benchmark (single-threaded) for "+str(RUNTIME)+" second(s)...")
	while (time() - start_time) < RUNTIME:
		#print("\r"+str(int((time()-start_time)*(100/RUNTIME)))+"%",end="")

		if (add):
			pi += (4/divider)
			add = False
		else:
			pi -= (4/divider)
			add = True
		
		divider += 2
		
	print ("\r100%")
	print("Benchmark complete!")
	print("Result : "+ str(divider) + " points.")
	running = input("Run again? (Y/N)")

 

 

benchmark.py

[themaniac]

i did 5 runs and got an average of 125.8, wow i got quite a shit cpu, CPU: E1-6010

[Fire Yoshi]

3 minutes ago, themaniac said:

i did 5 runs and got an average of 125.8, wow i got quite a shit cpu, CPU: E1-6010

Are you sure it wasn't 1258? 

[fizzlesticks]

Printing things while benchmarking is a horrible idea (unless you're trying to benchmark printing I guess.)

 

With the print: 23,000

Without it: 4,313,467

i5-2500k

[themaniac]

1 minute ago, Fire Yoshi said:

Are you sure it wasn't 1258? 

i'm sure it was 125.8

[Fire Yoshi]

7 minutes ago, fizzlesticks said:

Printing things while benchmarking is a horrible idea (unless you're trying to benchmark printing I guess.)

 

With the print: 23,000

Without it: 4,313,467

i5-2500k

The impact is indeed massive 

But I like having some kind of progress indicator, and yeah, I guess it does benchmark printing at the same time (maybe more than the actual calculation...)

So feel free to comment/uncomment the print in the loop (I edited it in my post), here are my scores without it :

Intel Core i5-4590T : ~ 4.85M

Intel Atom Z3560 : ~ 0.226M

[Fire Yoshi]

10 minutes ago, themaniac said:

i'm sure it was 125.8

Try it again with the print line commented (I edited it in the post)

[Gachr]

Did you remove the printing now? I ran it 3 times and got 3 million points on my Samsung laptop, I will probably benchmark my desktop and ASUS laptop tomorrow.

[Fire Yoshi]

16 hours ago, Gachr said:

Did you remove the printing now? I ran it 3 times and got 3 million points on my Samsung laptop, I will probably benchmark my desktop and ASUS laptop tomorrow.

I did, the line is now commented (with a " # " in front of it)

[Nineshadow]

C++ version :
 

#include <bits/stdc++.h>
long long t=1;
bool k=false;
double a=4;
int main()
{
    std::chrono::seconds time(1);
    auto start = std::chrono::high_resolution_clock::now(),cur=start;
    while(cur-start<=time)
        t+=2,a+=4.0/(k?t:-1*t),k=!k,cur=std::chrono::high_resolution_clock::now();
    std::cout<<t;
    return 0;
}

The result on my Lenovo Y50 (i7 4710HQ) is 20866449. On your Python version I got 4652623

[Gachr]

The results I got on my desktop were pretty strange... between 4.7 and 5.6 million points.

[Hamosch]

5 hours ago, Nineshadow said:

The result on my Lenovo Y50 (i7 4710HQ) is 20866449. On your Python version I got 4652623

I'll do you one better 

#include <bits/stdc++.h>
#include <x86intrin.h>
long long t=-1;
int main()
{
	std::chrono::seconds time(1);
	auto start = std::chrono::high_resolution_clock::now(),cur=start;
	double p[4];
	__m256d a = _mm256_setzero_pd();
	__m256d k = _mm256_set1_pd(4.0);
	__m256d b = _mm256_set_pd(1.0, -3.0, 5.0, -7.0);
	__m256d c = _mm256_set_pd(8.0, -8.0, 8.0, -8.0);
	while(cur-start<=time) {
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		a = _mm256_add_pd(a, _mm256_div_pd(k, b));
		b = _mm256_add_pd(b, c);
		t+=64;
		cur=std::chrono::high_resolution_clock::now();
	}
	a = _mm256_hadd_pd(a, _mm256_permute2f128_pd(a, a, 1));
	a = _mm256_hadd_pd(a, a);
	_mm256_store_pd(p, a);
	std::cout<<p[0]'\n';
	std::cout<<t<<'\n';
	return 0;
}

loop unrolled and with vector instrcutions compiled with -O3 and mavx:

 

original python:       6,000,000

optimized c++:    810,000,000

 

still only running on one core on a stock 4770k.

[Gachr]

1 hour ago, Hamosch said:

<snap>

 

I'll do you one better 

loop unrolled and with vector instrcutions compiled with -O3 and mavx:

 

original python:       6,000,000

optimized c++:    810,000,000

 

still only running on one core on a stock 4770k.

Interesting... We could try to "compare" languages with that algorithm...

[Gachr]

runtime = 1
running = 'y'

while running.casecmp('y').zero?
  start_time = Time.new.to_i
  divider = 1
  pi = 0
  add = true
  puts "Running Ruby benchmark (single-threaded) for #{runtime} second(s)..."

  while Time.new.to_i - start_time < runtime
    if add
      pi += 4 / divider
      add = false
    else
      pi -= 4 / divider
      add = true
    end
    divider += 2
  end
  puts 'Benchmark complete!'
  puts "Result: #{divider} points."
  puts 'Run again? (Y/N)'
  running = gets.chomp
end

Here is my Ruby code. It doesn't work very well there, I got results ranging from 1.2m to 5.8m.

Stay put for more langs

[vorticalbox]

my 8350 stock  on ubuntu does 

 

Running Pithon benchmark (single-threaded) for 1 second(s)...
100%
Benchmark complete!
Result : 4481769 points.
Run again? (Y/N)y

also typing "y" (lowercase) causes it to crash

[Hamosch]

30 minutes ago, Gachr said:

Here is my Ruby code. It doesn't work very well there, I got results ranging from 1.2m to 5.8m.

Stay put for more langs

Well my C++ code is not really fair as I'm using both loop unrolling and the Intel AVX (Advanced Vector Extensions) to get comparably very high throughput by completing 8*4=32 iterations (or adding 64 to the counter) each loop.

Loop unrolling increases the throughput because there are not as many branches and with AVX I can exploit the parallelism in the problem to execute 4 instructions in one in a SIMD way.

Everyone else uses a serial approach and no unrolling so there's no question to as why my code is faster regardless of being C++.

 

I really just wanted to see what kind of performance I could cram out compared to the naive solution.

But have fun, I just wouldn't compare my results of 130 times faster to the others all using the same approach.

[vorticalbox]

var pi;
var start =0;
var time;
var running= "y";
var n=1;

function calPi(p)
{
	if(start == 0 && running == "y")
	{
		start = new Date().getTime();
	}
	
	pi = pi+(4/n)-(4/(n+2));
	n+=4;
	
	time = new Date().getTime();
	if((time - start) >= 1)
	{
		running = "n";
	}else
	{
		calPi(n);
	}
}

calPi(1);
console.log(n);

ctrl+shift+i and paste it in console. I'm getting about 6 k on my 8350

[Gachr]

1 hour ago, vorticalbox said:

var pi;
var start =0;
var time;
var running= "y";
var n=1;

function calPi(p)
{
	if(start == 0 && running == "y")
	{
		start = new Date().getTime();
	}
	
	pi = pi+(4/n)-(4/(n+2));
	n+=4;
	
	time = new Date().getTime();
	if((time - start) >= 1)
	{
		running = "n";
	}else
	{
		calPi(n);
	}
}

calPi(1);
console.log(n);

ctrl+shift+i and paste it in console. I'm getting about 6 k on my 8350

JavaScript?

 

//Edit: Just noticed the console.log(), thanks for the version.

Anyways, it's screwed up, I get results from 5 (yes, the single digit 5) to about 12k.

[vorticalbox]

2 hours ago, Gachr said:

JavaScript?

 

//Edit: Just noticed the console.log(), thanks for the version.

Anyways, it's screwed up, I get results from 5 (yes, the single digit 5) to about 12k.

ysh I've noticed that too. I have never really messed with system time so its likely not set up correctly. I will keep editing and see if I can get it better.

[fizzlesticks]

14 minutes ago, vorticalbox said:

ysh I've noticed that too. I have never really messed with system time so its likely not set up correctly. I will keep editing and see if I can get it better.

getTime() is in milliseconds so you would need to do if((time - start) >= 1000)

However then it runs too long and crashes due to too much recursion.

[vorticalbox]

26 minutes ago, fizzlesticks said:

getTime() is in milliseconds so you would need to do if((time - start) >= 1000)

However then it runs too long and crashes due to too much recursion.

maybe do 2 500 ms bursts?

[fizzlesticks]

13 minutes ago, vorticalbox said:

maybe do 2 500 ms bursts?

Would still be way too much. You'll have to get rid of the recursion.

[vorticalbox]

18 minutes ago, fizzlesticks said:

Would still be way too much. You'll have to get rid of the recursion.

seems there is a 10,000 recursion limit

 

[Fire Yoshi]

7 hours ago, Gachr said:

Interesting... We could try to "compare" languages with that algorithm...

Well, I didn't make it to be efficient 

Took me maybe 10-15 mins out of pure boredom...

I'm pretty sure it could be optimized a lot more, even in Python