# Details on the July 29 and August 1 downtime

> Asana had downtime twice in the past week, 2 hours on July 29 and 50 minutes on August 1. We are continuing investigation, but want to let you know what we understand so far, we have made and are making to prevent a recurrence in the short-term.

Source: https://asana.com/inside-asana/july-29-august-1-downtime

## Details on the July 29 and August 1 downtime

Asana had downtime twice in the past week, 2 hours on July 29 and 50 minutes on August 1. We are continuing investigation, but want to let you know what we understand so far, and what changes we have made and are making to prevent a recurrence in the short-term.

Both of these incidents were caused by high database contention. This contention occurred on a database that is shared across all teams and workspaces (e.g. user authentication methods, like OAuth applications). We’ve been working on breaking up our monolithic application into services, and one of our goals with that project is to replace this database with a more scalable distributed data store. That work is ongoing, and currently we do writes to this database for objects that are shared across teams and workspaces.

In the short term, we’ve reduced the write frequency for a property that appears to have triggered the start of both events (the OAuth application last accessed time, which was by far the most frequently updated property). We’ve also fixed some bugs and modified database configuration to reduce deadlocks/contention, and updated our emergency response procedures to improve how we recover from such outages in the future.

We are committed to Asana being a service that you can consistently rely on, and this week did not meet our standards. We’ll provide a follow-up post with more information about what happened as soon as the investigation is complete, along with what we plan on doing to address this class of problem in the future (both near- and long-term).

- [Improving Asana’s Pageload Performance](/inside-asana/improving-asanas-pageload-performance)

Engineering

#### Senior Engineering Manager

Asana serves 2.5 million pageloads a day, peaking at 3.5 million. Each one is a user's first impression of the product — the moment between intent and action — and for a tool team ...

- [How Asana Built A Resilient ID Allocation System](/inside-asana/how-asana-built-resilient-id-allocation)

Engineering

Every object in Asana—every task, project, comment, and attachment—needs a unique identifier. At Asana, these IDs are sequentially incrementing integers, allocated in blocks from ...

- [The Athletic Position](/inside-asana/the-athletic-position)

Engineering

I. Before the Ball Leaves their StringsThere is a moment from the 1988 French Open final that most people who talk about tennis haven’t had to think about carefully.Steffi Graf pl ...

- [How Asana leverages AWS Inspector for total visibility over infrastructure vulnerabilities](/inside-asana/asana-leverages-aws-inspector-visibility-infrastructure-vulnerabilities)

Engineering

#### Software Engineer

Scanning for vulnerabilities across multiple AWS accounts, eliminating noise, and turning vulnerability findings into actionable work is a challenging but important undertaking. A ...

- [Details on the July 29 and August 1 downtime](/inside-asana/july-29-august-1-downtime)

Engineering

Asana had downtime twice in the past week, 2 hours on July 29 and 50 minutes on August 1. We are continuing investigation, but want to let you know what we understand so far, and ...

- [Engineering](/inside-asana/engineering-spotlight)