[WinAPI] Changing window style on the fly

[badreg]

I have two windows that are overlapped. The top window is a semi-transparent overlay created by another application and is the same size and position of the bottom window. I want to toggle which window I am interacting with on the fly. Windows API enables a window to have mouse transparency if it has an extended style of WS_EX_TRANSPARENT.

 

I am able to achieve this with the following code:

if (!strcmp(argv[1], "-O"))
   SetWindowLongPtr(hwnd, GWL_EXSTYLE, WS_EX_LAYERED);
else if (!strcmp(argv[1], "-T"))
   SetWindowLongPtr(hwnd, GWL_EXSTYLE, WS_EX_LAYERED | WS_EX_TRANSPARENT);

And an AutoHotKey script to call the application with a hotkey:

~LCtrl::RunWait, foo.exe -O,,Hide
~LCtrl Up::RunWait, foo.exe -T,,Hide

In the current implementation, the target window is mouse transparent unless I hold Ctrl. Once I release Ctrl, the WS_EX_TRANSPARENT style is added back, and it becomes mouse transparent again. This works, but foo.exe must be run as administrator to work because the top window is created by another process. If it is run normally, SetWindowLongPtr() returns 0 with an error of ERROR_ACCESS_DENIED.

 

I want to make this work without requiring adminstrator privileges and without AHK. I know that it is possible to do this with hooks and subclassing another process, but this is not an area that I am familiar with. Can someone point me in the right direction? Thank you!

[Franck]

If your window is started by another process you will never be able to do it. The other process has the handle of the window and you will never have access to change it. You can change down the process line but never up.

[badreg]

15 hours ago, Franck said:

If your window is started by another process you will never be able to do it. The other process has the handle of the window and you will never have access to change it. You can change down the process line but never up.

I'm not sure that this is correct, because I am already able to do this with the method summarized above. What I am trying to do is contain this functionality entirely within my application and be able to run it without administration privileges.

[Franck]

8 hours ago, badreg said:

I'm not sure that this is correct, because I am already able to do this with the method summarized above. What I am trying to do is contain this functionality entirely within my application and be able to run it without administration privileges.

if you can actually change it the process that started doesn't have the handle. hence it's not what i mentioned. But still if you are getting an ERROR_ACCESS_DENIED on the win32 call of SetWindowLongPtr it mean you have another process that has your handle which restrict you access.

 

** edit ** or you have a stuck handle. right now it seems Autohotkey is the master process. I bet if you were to call with a normal batch command line the same exe window and pass the same parameter that it will work.

[badreg]

18 hours ago, Franck said:

if you can actually change it the process that started doesn't have the handle. hence it's not what i mentioned. But still if you are getting an ERROR_ACCESS_DENIED on the win32 call of SetWindowLongPtr it mean you have another process that has your handle which restrict you access.

 

** edit ** or you have a stuck handle. right now it seems Autohotkey is the master process. I bet if you were to call with a normal batch command line the same exe window and pass the same parameter that it will work.

foo.exe works if I run it as administrator, and fails if I run it normally, so AHK is not what is making it work. In fact, AHK's WinSet command fails on this particular window (returns an ErrorLevel of 1), even though WinSet works on pretty much every other window.

 

How do I find out which process owns the handle? The window that is spawned by the external application is parented and owned by a window by my application. It follows the size and position of my window and destroys itself when I close my window. Based on this behavior, it seems like the spawning application is still controlling its own window.