WebDriver For Journalists: Scraping the Web To Report the Truth
10 Oct 2023
-
Locked
Did you know that in 2021, a Pulitzer Prize was awarded to a project that had WebDriver code at its core?
The New York Times COVID data-tracking project became the United States' most-watched dashboard for tracking changes in the spread of the pandemic. It worked by aggregating data from municipalities across the nation. These sources ranged in sophistication from sanitized data available for download, to bespoke HTML maps.
In this presentation, we'll discuss the role of WebDriver and other web-crawling technologies in that and other journalistic endeavours.Â
We'll review using selectors to find data for journalists, cleaning source data, and the value of agility in deadline-driven workflows.Â
We'll also explore how the lessons learned in this line of work are applicable to the practice of software testing and beyond.
The New York Times COVID data-tracking project became the United States' most-watched dashboard for tracking changes in the spread of the pandemic. It worked by aggregating data from municipalities across the nation. These sources ranged in sophistication from sanitized data available for download, to bespoke HTML maps.
In this presentation, we'll discuss the role of WebDriver and other web-crawling technologies in that and other journalistic endeavours.Â
We'll review using selectors to find data for journalists, cleaning source data, and the value of agility in deadline-driven workflows.Â
We'll also explore how the lessons learned in this line of work are applicable to the practice of software testing and beyond.
Better than a generic video, see YOUR test, live, ready to show you what matters most: quality at scale.
Explore MoT
What I learned about influence by becoming a stakeholder
Gain the tools you need to become an operability advocate. Making your testing even more awesome along the way!
Into the MoTaverse is a podcast by Ministry of Testing, hosted by Rosie Sherry, exploring the people, insights, and systems shaping quality in modern software teams.
Comments