Sponsors

<-- Back to schedule

Building data flows with Celery and SQLAlchemy

Project: Celery (canvas in particular), SQLAlchemy

Reporting and analysis systems rely on coherent and reliable data, often from disparate sources. To that end, a series of well established data warehousing practices have emerged to extract data and produce a consistent data store.

This talk will look at some options for composing workflows using Python. In particular, we'll explore beyond Celery's asynchronous task processing functionality into its workflow (aka Canvas) system and how it can be used in conjunction with SQLAlchemy's architecture to provide the building blocks for data stream processing.

Roger Barnes

Roger is an experienced cross-functional software developer specialising in web application development and deployment across the entire application lifecycle.

He has run a web development business, been a key driver of internal enterprise deployments at a large enterprise software vendor, cofounded a tech startup in the travel industry and is currently doing Python-specific contract work.

A keen and constant learner, Roger has contributed to open source projects and communities as a user, advocate and speaker.