A weird phenomenon that I can't explain

[byubi]

Hi, I chose to post my problem here since there are a lot of ppl here that know about graphics and stuff, and since it's not a "traditional" programming problem. It's a bit long so bare with me.

 

I made a program that takes a screenshot of your screen while a game is open, scans that image looking for some text and numbers, then do other stuff with those information (calculations, logging...). PS: nothing that violates the EULA of the game.

 

How the program works: it takes the screenshot and converts it into a matrix where every element represents a pixel, then compares that to my database to try to find a match.

 

Hundreds of ppl use the program and it works fine for them, because the way the game/system renders the frame: the pixels that make a certain letter/word are laid out exactly the same way on every user's screen... EXCEPT FOR 2 USERS !!!

 

Only 2 users have reported this problem. They sent me a screenshot, for the naked-eye the text looks no different from everyone else's, but when you look at the pixels that make that text they are different from what the other hundreds of users have !

 

Anyone have an explanation for this? is it possible that this is caused by a specific graphics card or driver?

[ImBleu]

Thats.. different to say the least.

 

How strict is your program? does it allow for minor leeway at all in the shapes of letters?

[MrKickkiller]

Well, perhaps they were in a scenario where a neigboring pixel to the text has the exact same colour as the text.

Therefor the text look pretty much the same, but your program will recognize an odd letter. Lets say a T with an extra dot on it?

Then  the program would trow an error or exception and therefor causing trouble?

[Nuluvius]

This is image processing. How are you achieving it? are you using anything like OpenCV for example? I suggest that you really need to work on recreating the bug first in a controlled environment. You'll then obviously be able to debug and ascertain the cause. What are the specification differences in the edge cases? Introduce logging perhaps?

 

Things to consider:

• Internationalization
• Code pages
• OS font settings

Really we need more information. If you are unwilling to provide it then that would tend to make one suspicious of illicit activity. You mention 'hundreds of people' using the software...

[byubi]

Thank you guys for your input. To answer your questions :

- No, I don't use OpenCV or any other recognition software/API, I wrote everything from scratch.

-The image i'm processing is the one that windows puts in your clipboard when you press print-screen.

-The program was originality 100% strict because 1) the text I'm parsing has a uni-color background so no interfering pixels there 2) it was working for everyone, except for those 2 damn users :-/

I can easily make the parsing more flexible so that the text will be readable even with a few pixels off, and that's what i'm gonna do... no problem there.

But I just wanna understand why is it different for some people? In theory it should be the same right? If the letter 'l' is 10 pixels in height on my screen, it should be 10 pixels on every screen, right?

The fact that it works fine for 100s of users on many different platforms tells me that there is something wrong (or lets just say different) about those 2 users.
 

 

Things to consider:

• Internationalization
• Code pages
• OS font settings

 

These make sens to me, I will look into these. Thank you.

[Nuluvius]

...

But I just wanna understand why is it different for some people? In theory it should be the same right? If the letter 'l' is 10 pixels in height on my screen, it should be 10 pixels on every screen, right?

...

 

My gut feeling right now (without getting into any research) is not if their OS font/DPI (especially the DPI) settings are different on their systems. Run some experiments/conduct some research?

[WanderingFool]

I agree with the other people, the first thing that popped into my mind was that they might have a different font installed that makes it render slightly differently (like a modification of the font the game uses).

 

Other potential causes:

Screen resolution (If they have a funky screen res, it might force a slightly different rendering)

Hardware differences (who knows maybe their video card makes a floating point error and thus shifts it by one)

Is everything shifted by 1 or 2 pixels or just the text, because if you are capturing the whole screen, maybe something is pushing the game up by 1 pixel

[mikebald]

My initial thought is that those people are running the game at an obscure aspect ratio and that makes the game render the numbers different. An example of this is 1920x1080 vs 1920x1200 where it's 16:9 vs 16:10; you should be able to find if that's the case by looking at the dimensions of the screenshots.

 

Edit: I like the t-shirt in your avatar.

[Orcblood]

Can i has code? Or at least program? Or both?? I know nothing about this but would love to learn more. Sorry, I can't be help since this is a bit beyond me but wish you the best man  

[LukeTim]

Why are you matching against a database and just assuming it's the same every time? Some sort of OCR would be much more robust.

[WanderingFool]

Why are you matching against a database and just assuming it's the same every time? Some sort of OCR would be much more robust.

 

While I don't know the program that @byubi wrote, I would assume that OCR is overly complex to solve a simple problem.  I could make a calculator that uses base 10 decimals, but most people just go with floats or doubles because it is easier.

[LukeTim]

While I don't know the program that @byubi wrote, I would assume that OCR is overly complex to solve a simple problem.  I could make a calculator that uses base 10 decimals, but most people just go with floats or doubles because it is easier.

 

I don't think it's necessarily overly complex.

By the sounds of it, byubi is already doing some matching... just very precise matching that expects to see the exact same set of pixels each time. OCR just matches a little more cleverly so that slight differences are not an issue.

In fact, come to think of it, you may not even need a full OCR implementation. You could possibly do edge detection to distinguish the letters and match that pixel for pixel against the database. That would mean small changes in pixel values would no longer be an issue, because the edge detection would simply put out a black and white (0 or 255) image. You could possibly even save space that way since each pixel would then be representable by a single bit.

Not sure if that would really work... It's just an idea. It should work... It's been a long time since I've done any proper image processing though...

[byubi]

Hey, in case you guys are interested here's a link to the program page where you can find a short video demo : http://www.entropiaforum.com/fast-trade/

If I were to make this program now I would probably use an OCR, but at the time I started this project (many years ago) I was a n00b programmer and OCR seemed very complicated. I implemented my own home-made matching method, tried it on different machines and it worked. So I assumed it would work on all machines, and for years it did... until recently.

I'm going to introduce the Levenshtein distance into the matching function, it should work.

[Darko_Nerivar]

Also add logs in future releases

That will save you helluva lot of time with bug reports

 

PS yeah, it's boring, but somewhat rewarding