LightningCrawler.net A really fast http web crawler using .Net Core & SQL Server Run the console app in Crawler project And check your report in Dashboard project