Software - Making Data Mistakes

Making Data Mistakes

Sign in Subscribe

Software

A collection of 10 posts

How to make your software team much more effective

Programming is the closest we have to magic in the real world. Just like magic, you can conjure rich and powerful creations with the power of your own mind, given the right tools and training. Just as powerful magic could lay waste to an entire army, powerful programming can do

Make it work, make it right, make it fast

Make it work, make it right, make it fast

I first heard this phrase from Gael Varoquax [http://gael-varoquaux.info/] (one of the core scikit-learn committers) in a talk[1], and I loved the super-concentrated wisdom of it: * Make it work. First of all, get it to compile, get it to run, make sure it spits out roughly the

How to reproduce your analysis

How to reproduce your analysis

What do you do when things don’t seem to be behaving the same way now that they did before? Here are a few techniques to avoid that dread scenario.

How to Cheat at Data Science (with help from Centaurs and the Wizard of Oz)

What does it take to run a high-performing data science team? In this 15-minute video, I talk about two tools that I’ve found invaluable for doing just that: * Wizard-of-ozzing is a technique for product discovery that can be very helpful for evaluating potential problems to work on [6:00]

What makes a great data scientist and data science team? (Podcast)

I was recently honoured to be a guest on The Data Pubcast [https://anchor.fm/data-pubcast/episodes/The-Data-Pubcast--Pint-3-ek3l21/a-a3b42vj] - a podcast about making data accessible to everyone, hosted by the incredibly talented Nick Latocha and Andy Crossley. In the episode [https://anchor.fm/data-pubcast/episodes/The-Data-Pubcast--Pint-3-ek3l21/a-a3b42vj] we discuss:

How to write good software faster (we spend 90% of our time debugging)

How to write good software faster (we spend 90% of our time debugging)

If we spend the majority of our programming time and effort on debugging, we should focus our efforts on speeding up our debugging (rather than trying to write code faster).

How to know if your recommendations algorithm is actually doing a good job

I led the team that built Channel 4’s recommender system for All 4 in 2016. It started out as a straightforward project. But after getting lost in a rabbit hole trying to devise a score for ‘provocativeness’ and ‘serendipity’, I learned the single most important lesson about data science.

Why are my emails being sent to spam - a guide for GSuite

Why are my emails being sent to spam - a guide for GSuite

“This email to an investor was marked as spam! They only noticed it by accident. Why is this happening?”

The Difference: Throughput vs Latency

The Difference: Throughput vs Latency

Latency and Throughput are important concepts for data scientists. How are they distinct? And why is this distinction important, not just for technical systems, but also for team performance?

Technical debt isn't like financial debt

Technical debt isn't like financial debt

I hope you weren’t hoping for a blanket answer to the question of exactly when to pay off technical debt. If you were, the answer is Tuesday. Almost all interesting decisions are about risks and trade-offs. This is a framework for making our intuitions about technical debt more rigorous.