Under The Hood: The Technology Of Hoodline

Curious about the technology that powers Hoodline?

For the past three years, we've been designing and developing the tools to support a modern digital newsroom. Below, you'll find a quick introduction to some of our main products and the core services and solutions behind them.

Publishing System

Hoodline evolved from three individual neighborhood blogs which were originally published on WordPress. When it came time to unify them, we chose to build an integrated content management system for all of Hoodline.

Initially our CMS didn’t stray far from the template laid down by WordPress, but over time we customized it to better reflect our editorial structure and workflows.

Our CMS now supports multiple content types (like news articles and event listings) and formats (long-form stories, tweet-based mini-stories, captioned photos, and the like). It also allows us to embed event details from our own calendar into our stories in the same way you’d embed a tweet or Instagram photo into a webpage.

We’re continually iterating on the CMS, both in terms of features and the underlying architecture, with the goal of making it a user-friendly environment in which to write and edit stories, and an efficient platform for publishing news to devices of all kinds.

Local Data Feeds

Examining the many freely available data sets that city and state agencies publish is one of the most useful ways we come up with story ideas. Crime incidents, building permits, liquor license applications, business registrations and planning records are all invaluable sources of information about what's going on in the city.

There are a few problems with getting this data directly from the source, however. The various agencies publish the data in many different formats, from spreadsheets to web pages to raw database dumps.

So, we built a pipeline to pull that data from various sources into our system, clean it up, tag it by neighborhood and dump it into a single unified database.

Once the data's in our database, we can use an internal web tool to do full text searches, either across the entire city or filtered by specific neighborhoods.

We also send notifications to a dedicated Slack channel when we bring new data into our system, so that our writers and editors can get an immediate heads-up about potentially newsworthy activity.

Core Technology

These systems and other internal tools are built with the same core set of technologies:

  • Ruby on Rails - great tools for building web systems quickly and elegantly
  • React and Redux (JavaScript) - our 2015 election guide was just one example of the slick interactive web experiences we’re able to build with these modern front end web technologies
  • PostgreSQL and Elasticsearch - search and storage systems enabling geo- and full-text searching across data sets of various shapes and sizes
  • Redis - serves us well as a cache for transient data and pub/sub
  • Sidekiq - the backbone of our data pipeline and the hub for integrations with many external systems
  • Heroku - our web host, making it as easy to push changes to our website as pressing a single button
  • Imgix and Amazon CloudFront - content delivery networks that serve our photos and other resources
  • GitHub, CircleCI, Airbrake, Librato, Papertrail - if you know what these developer-oriented services do, you know why they’re so critical to our entire operation

One of our core values as a company is to remain at the forefront of innovation in local media. We also aim to be transparent with our readers. So, expect to see frequent updates here as our engineering team builds new features, overcomes thorny technical challenges, and keeps on innovating as Hoodline continues to grow.