Windows Task Scheduler Messing Up Python Script

[Slottr]

So I've found myself in a bit of a weird situation

 

I have a python script that's to run each night via Windows  task scheduler.

The script scrapes some data,  updates a txt file, and sends that information as a discord message. This is all within a local directory

 

When I run the file on its own, the values it sends in the discord are correct

 

However when I run it with the task scheduler, the values in the discord message are totally different, with missing information. However, the local text file is correct

 

Is there any reason why running the Task Scheduler would be screwing this up? Any ideas would be helpful at this point lol

 

 

[Slottr]

1 minute ago, Sakuriru said:

Under what user does the task scheduler run?

It's using the highest permissions

[Slottr]

18 minutes ago, Sakuriru said:

Is the file being referenced using absolute or relative paths (i.e., ~/documents/)?

All references are absolute paths 

[Pandur]

Does it really require highest permissions? Or can it be run with your user account? Running with higher permissions than necessary can also cause its own issues.

[Slottr]

14 minutes ago, Pandur said:

Does it really require highest permissions? Or can it be run with your user account? Running with higher permissions than necessary can also cause its own issues.

I’ve tried just about every level possible. Probably should have mentioned that in the initial post 

 

no matter which level of perm it has, same outcome 

[Pandur]

Ok. And this is one script that does every operation? Does it scrape local values or from an online source?

 

One idea would be to run the scraper and reporter as two separate modules in a main script. I would make three scripts. Main, plus one for each module. Run scrape module, save scraped data as dataframe object (or similar), run report module to send data from dataframe. Then you can of course save or append the scraped data to a text file. But unless there is a specific reason to save to file before reporting, you can do without that step, one less chef to mess up your dish.

 

Is this run while your user is logged in at night, wake from sleep task, or while at the lock screen?

 

If you still get wonky results, you need to debug your scraper with some known data. And the reporter, separately.

[Slottr]

40 minutes ago, Pandur said:

Ok. And this is one script that does every operation? Does it scrape local values or from an online source?

 

One idea would be to run the scraper and reporter as two separate modules in a main script. I would make three scripts. Main, plus one for each module. Run scrape module, save scraped data as dataframe object (or similar), run report module to send data from dataframe. Then you can of course save or append the scraped data to a text file. But unless there is a specific reason to save to file before reporting, you can do without that step, one less chef to mess up your dish.

 

Is this run while your user is logged in at night, wake from sleep task, or while at the lock screen?

 

If you still get wonky results, you need to debug your scraper with some known data. And the reporter, separately.

The code works fine, it’s not an issue with the program itself. Its an issue with the windows task scheduler, on a windows user who is constantly logged in.

[Pandur]

18 minutes ago, Slottr said:

The code works fine, it’s not an issue with the program itself. Its an issue with the windows task scheduler.

I understand that your code works fine when run manually. But when you hand your eggs over to the task scheduler and ask for chickens. You get scrambled eggs.

 

It sounds to me like you have been all over your task scheduler settings. Since we can't rewrite the code for the task scheduler service, and your code does not run properly when started with it, your code has to be considered the faulty part. Because that is what you can change.

 

So, that leaves debugging, restructuring, and rewriting your code to make it work.

 

I have run into similar-ish issues before. I have, on more than one occation, had to make two versions of the same script just to get around idiotic Python/OS limitations when running on different OS. So yeah, the script can work perfectly fine. Until you try to execute it in a different manner

[Slottr]

15 minutes ago, Pandur said:

I understand that your code works fine when run manually. But when you hand your eggs over to the task scheduler and ask for chickens. You get scrambled eggs.

 

It sounds to me like you have been all over your task scheduler settings. Since we can't rewrite the code for the task scheduler service, and your code does not run properly when started with it, your code has to be considered the faulty part. Because that is what you can change.

 

So, that leaves debugging, restructuring, and rewriting your code to make it work.

 

I have run into similar-ish issues before. I have, on more than one occation, had to make two versions of the same script just to get around idiotic Python/OS limitations when running on different OS. So yeah, the script can work perfectly fine. Until you try to execute it in a different manner

So here’s a chunk which may clear some stuff up to help me: 

 

I ran the code without scraping, just to read the file and print the data (so everything is local), and about 10 lines. 
 

Same faulty output. It’s like it’s taking in information from somewhere else, which makes no sense as everything to be read or written to uses an absolute path to the directory. 
 

The code I ran was basic file opening, reading and print statements. Similar or even 1:1 to anything you’d see on any tutorial site. 

[Slottr]

Fixed it.

 

Found a SO page that solved my issue after some pretty hard digging

 

https://stackoverflow.com/questions/20196049/problems-running-python-script-by-windows-task-scheduler-that-does-pscp

 

Solved by setting the "Start In" field to the directory of the python file.

[Pandur]

1 minute ago, Slottr said:

Fixed it.

 

Found a SO page that solved my issue after some pretty hard digging

 

https://stackoverflow.com/questions/20196049/problems-running-python-script-by-windows-task-scheduler-that-does-pscp

 

Solved by setting the "Start In" field to the directory of the python file.

Very good. Sometimes the devil is in the details.

[Slottr]

Just now, Pandur said:

Very good. Sometimes the devil is in the details.

Lol. Kind of a shitty place to write “(Optional)” when it doesn’t seem super optional. 

[Pandur]

3 minutes ago, Slottr said:

Lol. Kind of a shitty place to write “(Optional)” when it doesn’t seem super optional. 

Yeah. There are plenty of use cases where it is absolutely not required. But for running something to work with files in a specific folder it is 100% required. Problem solved for now at least.

[El Capitan Sponge]

@Slottr I have had similar issues with PHP in the past so at least you should know that this issue can affect any language and not just python