Time needed: 5 minutes
PowerShell is an amazing tool that can be used to perform various tasks on local and remote Windows desktops and Servers.
In addition to this, PowerShell is capable of performing numerous additional practical tasks that might be of interest to many Windows users. One of these tasks is called web scraping.
Using PowerShell, web scraping can be performed very quickly and efficiently. In below steps we will explore the commands that will help with scraping web data.
- Launch PowerShell
Click Start –> Search for PowerShell and click the PowerShell application to launch it
- Issue Web Request Command
We will scrape data from https://www.cybersecuritylearning.net.
We will store the scraped data in a variable called $cybersecurity
PS C:> $cybersecurity = Invoke-WebRequest https://www.cybersecuritylearning.net
PS C:> $cybersecurity
- $cybersecurity.Links
$cybersecurity.Links commands will provide all types of links in the website and will give an output that is not usable in general.
- Export to CSV for reporting
The result of the previous step may be extremely large for your own site, therefore it is easier to simply export the data to a csv and report on the desired information using Excel or other data tools.
Run the following command to output to a CSV file:
PS C:> $cybersecurity.Links | Export-Csv -Path E:\cybersecurity.csv -NoTypeInformation
Note: Change E:\cybersecurity.csv to your desired location and file name.
The output of the above command is as follows:
Leave A Comment