WebDriver For Journalists: Scraping the Web To Report the Truth
10 Oct 2023
-
Locked
Did you know that in 2021, a Pulitzer Prize was awarded to a project that had WebDriver code at its core?
The New York Times COVID data-tracking project became the United States' most-watched dashboard for tracking changes in the spread of the pandemic. It worked by aggregating data from municipalities across the nation. These sources ranged in sophistication from sanitized data available for download, to bespoke HTML maps.
In this presentation, we'll discuss the role of WebDriver and other web-crawling technologies in that and other journalistic endeavours.Â
We'll review using selectors to find data for journalists, cleaning source data, and the value of agility in deadline-driven workflows.Â
We'll also explore how the lessons learned in this line of work are applicable to the practice of software testing and beyond.
The New York Times COVID data-tracking project became the United States' most-watched dashboard for tracking changes in the spread of the pandemic. It worked by aggregating data from municipalities across the nation. These sources ranged in sophistication from sanitized data available for download, to bespoke HTML maps.
In this presentation, we'll discuss the role of WebDriver and other web-crawling technologies in that and other journalistic endeavours.Â
We'll review using selectors to find data for journalists, cleaning source data, and the value of agility in deadline-driven workflows.Â
We'll also explore how the lessons learned in this line of work are applicable to the practice of software testing and beyond.
With servers in >250 cities around the world, check your site for localization problems, broken GDPR banners, etc.
Explore MoT
Thu, 1 Oct
A tech conference to help you navigate the ever-shifting landscape of Quality Engineering, AI, Leadership, Product, Accessibility and Security.
Gain the tools you need to become an operability advocate. Making your testing even more awesome along the way!
Debrief the week in Quality via a community radio show hosted by Simon Tomes and members of the community
Comments