Webpage scraping with Powershell

[Helpful Tech Witch]

Im looking for a way to find the most recent download link for the AMD adrenalin drivers for my RX580.

I know that i can scrape all of the links to the download page with 

(Invoke-WebRequest –Uri ‘https://www.amd.com/en/support/graphics/radeon-500-series/radeon-rx-500-series/radeon-rx-580’).links

but am looking for a way to sort all of the links to find all of them from https://drivers.amd.com/drivers/, then sort for all links containing win10-64 in them.

Then id like to save it to something like 

$AmdDriver

for use in another script

 

How would I go about this?

[shadow_ray]

... | Select-String  -Pattern "win10-64"

[Helpful Tech Witch]

30 minutes ago, shadow_ray said:

... | Select-String  -Pattern "win10-64"

Ok, yep, that worked. 

Now, i get two options back, with this as the output

@{innerHTML=Download*; innerText=Download*; outerHTML=<A class=btn-transparent-black 
href="https://drivers.amd.com/drivers/non-whql-radeon-software-adrenalin-2020-21.5.2-win10-64bit-may17.exe">Download*</A>; outerText=Download*; tagName=A; class=btn-transparent-black; 
href=https://drivers.amd.com/drivers/non-whql-radeon-software-adrenalin-2020-21.5.2-win10-64bit-may17.exe}
@{innerHTML=Download*; innerText=Download*; outerHTML=<A class=btn-transparent-black 
href="https://drivers.amd.com/drivers/radeon-software-adrenalin-2020-21.4.1-win10-64bit-apr20.exe">Download*</A>; outerText=Download*; tagName=A; class=btn-transparent-black; 
href=https://drivers.amd.com/drivers/radeon-software-adrenalin-2020-21.4.1-win10-64bit-apr20.exe}

with the links being "https://drivers.amd.com/drivers/non-whql-radeon-software-adrenalin-2020-21.5.2-win10-64bit-may17.exe" and "https://drivers.amd.com/drivers/radeon-software-adrenalin-2020-21.4.1-win10-64bit-apr20.exe"

 

I know i need to store them some how, and check for the version number being the greatest, via something like checking if number is greater one by one, then using the link with the higher number. 

 

id probably need to save link one is $DriverOpt1 and $DriverOpt2 search each for a somethign in the form of -##.#.#-, where # is any number, then check which is the greatest

 

How wouldf I do that? (sorry ive been learniong most of what I can do with google, i cant seem to find how to do this)
 

[shadow_ray]

8 hours ago, HelpfulTechWizard said:

ive been learniong most of what I can do with google,

so do I

9 hours ago, HelpfulTechWizard said:

search each for a somethign in the form of -##.#.#-, where # is any number, then check which is the greatest

Regex.

$found = $string -match "-\d{4}-\d+\.\d+\.\d+-win10-64bit"

$matches[0]

 

where

"-" match a dash

"\d{4}" match 4 decimal digits

"\d+" match 1 or more digits

"\." match a single period

$found == true when there is at least one mach

$matches contains matched strings

 

 

[Helpful Tech Witch]

16 hours ago, shadow_ray said:

so do I

Regex.

$found = $string -match "-\d{4}-\d+\.\d+\.\d+-win10-64bit"

$matches[0]

 

where

"-" match a dash

"\d{4}" match 4 decimal digits

"\d+" match 1 or more digits

"\." match a single period

$found == true when there is at least one mach

$matches contains matched strings

 

 

Ok, the syntax is way too complicated for me to figure out how to implement it, but I was able to figure it out,at least mostly.

The newest driver version is always non-whql, which is reflected in the URL. So, by using 

$Driver = (Invoke-WebRequest –Uri ‘https://www.amd.com/en/support/graphics/radeon-500-series/radeon-rx-500-series/radeon-rx-580’).links.href | Select-String  -Pattern "win10-64" | Select-String -Patern "non-whql"

It will get only the new one.

I cant send it to my dowload yet though, as it has a quote at the beginning and end of the link, so Invoke-WebRequest can't figure it out.

 

Any guess on how to get Invoke-WebRequest to parse it?

[shadow_ray]

7 hours ago, HelpfulTechWizard said:

I cant send it to my dowload yet though, as it has a quote at the beginning and end of the link, so Invoke-WebRequest can't figure it out.

 

Any guess on how to get Invoke-WebRequest to parse it?

 

$href = $href.substring(1,$href.length -2)

 

This will remove the first and last characters "hello" >> "ell"

[Helpful Tech Witch]

12 hours ago, shadow_ray said:

 

$href = $href.substring(1,$href.length -2)

 

This will remove the first and last characters "hello" >> "ell"

Powershell says i dont have a command for .substring

Method invocation failed because [Microsoft.PowerShell.Commands.MatchInfo] does not contain 
a method named 'substring'.
At C:\Users\urban\Downloads\amd driver.ps1:2 char:1
+ $Uri = $Driver.substring(1,$Driver.length -2)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : MethodNotFound
 
Invoke-WebRequest : Cannot validate argument on parameter 'Uri'. The argument is null or 
empty. Provide an argument that is not null or empty, and then try the command again.
At C:\Users\urban\Downloads\amd driver.ps1:5 char:24
+ Invoke-WebRequest -Uri $Uri -OutFile $output
+                        ~~~~
    + CategoryInfo          : InvalidData: (:) [Invoke-WebRequest], ParameterBindingValidat 
   ionException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Command 
   s.InvokeWebRequestCommand

 

[shadow_ray]

$Uri is probably not a string then.

What's the result from $Uri.GetType() ?

[Helpful Tech Witch]

2 minutes ago, shadow_ray said:

$Uri is probably not a string then.

What's the result from $Uri.GetType() ?

im making $Uri a string, what im making it have is 
 

$Uri = $Driver.substring(1,$Driver.length -2)

to set the string $Uri to ($Driver=first and last)

When I run that, its a null-valued expression, as the script can never set a value to anything

[shadow_ray]

10 minutes ago, HelpfulTechWizard said:

im making $Uri a string, what im making it have is 
 

$Uri = $Driver.substring(1,$Driver.length -2)

to set the string $Uri to ($Driver=first and last)

When I run that, its a null-valued expression, as the script can never set a value to anything

Ohh sorry I switched up $Uri and $Driver in my mind.

What i meant to say was that $Driver is probably not a string so it doesn't have the substring function.

[Helpful Tech Witch]

10 minutes ago, shadow_ray said:

Ohh sorry I switched up $Uri and $Driver in my mind.

What i meant to say was that $Driver is probably not a string so it doesn't have the substring function.

It should be, its set right before it:

$Driver = (Invoke-WebRequest -Uri "https://www.amd.com/en/support/graphics/radeon-500-series/radeon-rx-500-series/radeon-rx-580").Links.Href | Select-String  -Pattern "win10-64" | Select-String  -Pattern "non-whql"

heres the script im working with rn

amd driver.ps1

[Trilithium]

This adapted script snippet worked for me doing the download (to the users temp folder):

$webUri = "https://www.amd.com/en/support/graphics/radeon-500-series/radeon-rx-500-series/radeon-rx-580"
$wr = Invoke-WebRequest -Uri $webUri -useBasicParsing
$drvUri = ($wr.links.href | Select-String -Pattern "win10-64" | Select-String -Pattern "non-whql" | Select-Object -First 1).tostring()
$filename = $drvUri.split("/") | Select-Object -Last 1
$s = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$s.Headers.add("Referer",$webUri)
$dr = Invoke-WebRequest -Uri $drvUri -OutFile "$env:temp\$filename" -WebSession $s -UseBasicParsing
Start-Process -FilePath "$env:temp\$filename"

Noteworthy modifications are:

• convert the driver URI from the object you receive from the selects to a text via .tostring().
• The AMD page requires a referer for downloading the driver - that's why I've added the Webrequest session object.