Example: collecting smart contract address from the State of Dapps
Web scraper can be add through chrome extension store. This article gives an example scraping data that hidden in links and scraping in multiple pages.
Open the website using chrome that containing useful data.
Open developer tools or press Ctrl + Shift + i on keyboard. Click on “Web Scraper” on the navigation bar.
Click on “Create new sitemap”, add name and starting URL. If you need to collect data from multiple pages, use [number1-number2] to specify page number in the URL.
Click on “Sitemap your_site_map_name - Create selector” and create a new selector.
Id & Type: Input Id and type depends on what kind of data you are selecting. Here I want to scrape the names and the contract address that hide in the name URL, so I chose “Link”.
Selector: Click on “Select”, press Alt, left click on the element you want, it will automatically select same format of element in Html. Then, click “Done selecting”, remember select “Multiple” checkbox below.
You can check the information selected by clicking “Data preview”.
Parent Selector: Here is to assign where your selector located. This “Name” selector in the example is obviously at the home URL, so it belongs to “_root”.
Parent Selector: Now, inside the “Name” link. we select the smart contracts’ address, because it’s wrapped in outer URL, this selector’s parent will be “Name”.
Selector map: Click on “Sitemap your_site_map_name - Selector map” and create a new selector, and you can show the structure of your selectors.
Start scraping: Click on the “Sitemap your_site_map_name - Scrap” to start scraping. You could export all the data as a CSV file after it finished.