Skip to content

Release Rollout Health

Release rollout health provides early guidance about the crash health of an application's current release. It appears next to release adoption on application cards, Favorites, List view and in the release summary at the top of each application page.

Adoption shows how widely a version is being used among reported sessions. Health answers a different question: does the available crash evidence look reassuring, concerning, or still uncertain?

Select the health badge to open the application's crash-free sessions view for that release.

What data is assessed?

The assessment uses sessions for the application's configured store bundle identifier, starting from the current release's release date and covering at most the last seven days.

The assessment also considers how many distinct devices are reporting and how many have been affected, so repeated crashes on one device alone do not trigger a problem verdict.

When comparing releases, Appelium uses the preceding released version of the same application and its sessions during the same assessment period, not its lifetime crash rate.

Why confidence matters

A release with no crashes in a few sessions has not demonstrated the same reliability as one with no crashes in thousands of sessions. Similarly, a small number of crashes can make a small sample look much worse than it really is.

Appelium therefore considers uncertainty as well as the measured crash-free percentage. With limited data, the evidence must clearly support a verdict; a simple comparison with a threshold is not enough. This is why a release with a seemingly good percentage may still show Gathering evidence.

For larger samples of at least 5,000 assessed sessions across 1,000 reporting devices, the measured crash-free rate settles cases close to the healthy threshold. The comparison used to detect a regression still accounts for uncertainty.

The four health levels

Healthy so far

The evidence supports a crash-free session rate of 99.9% or higher, matching the Dashboard's Good or Excellent levels, with at least 100 distinct reporting devices. For smaller samples, confidence in meeting that threshold is required; larger samples use the rule described above.

  • A previous release is not required.
  • Zero crashes alone is not enough when the sample is small.
  • A release meeting the healthy criteria is not flagged as a regression merely because the previous version was even better.

What to do: Treat this as a reassuring crash signal and continue monitoring as adoption grows. It is not a guarantee that the release is free of other issues.

Gathering evidence

There is not enough clear, recent evidence for another verdict. This can happen when:

  • Too few sessions or devices have reported.
  • The result is too close to the healthy threshold to distinguish reliably.
  • A concerning crash rate has affected fewer than three distinct devices.
  • No session occurred in the last 24 hours of the assessment period.

There is no fixed waiting time. Smaller applications can take longer to accumulate evidence and may remain in this state because the assessment only covers up to seven days. It does not mean the release is healthy or that a regression has been detected.

What to do: Allow more evidence to arrive, and review any reported crashes rather than interpreting this state as approval to expand a rollout.

Investigate

The evidence indicates that crash-free sessions are below 99.9%, with crashes affecting at least three distinct devices, but a meaningful deterioration from the previous release has not been established.

This includes releases whose crash health is similar to an already-unhealthy previous version, as well as cases where the previous release has no usable data or too little evidence for a reliable comparison.

What to do: Review the release's crashes. The absolute crash health is concerning even though a new regression has not been confirmed.

Regression detected

The release meets the same problem criteria as Investigate, and its crash rate is convincingly worse than the previous release after accounting for uncertainty in both samples.

The deterioration must clear both a 50% relative increase and a 0.05 percentage-point increase in crash rate. A small difference between displayed percentages is not enough on its own.

What to do: Prioritize investigating the affected release and consider pausing further rollout while you assess the impact.

Updates and limitations

  • The assessment runs periodically and reflects recent session data; it does not describe activity in real time.
  • The badge appears once an assessment is available for the current release. No badge does not mean the release is healthy.
  • If app is not frequently used, the Dashboard can continue showing the last available result. A badge is not a guarantee that fresh data is arriving.
  • Rollout health currently assesses crashes, not startup speed, network reliability or overall application quality. It is guidance for your investigation and does not automatically pause or advance a store rollout.