Why airline rankings disagree on the same carrier in the same year

Airline rankings disagree because they measure different variables. Skytrax’s passenger-voted awards capture traveler sentiment, while its star ratings flow from a professional audit of 500 to 800 service-delivery items.

AirlineRatings.com scores product and financial criteria against an editorial checklist; Cirium tracks one operational fact — how often flights reach the gate within 15 minutes of schedule. None is a substitute for the others.

To read a ranking honestly, first ask which question the number was built to answer.

Two respected ratings can look at the same airline in the same year and return opposite verdicts. It happens every award season across Asia-Pacific, and travelers are left assuming one of the raters slipped up.

The real explanation is duller but more useful: the raters are not measuring the same flight. Skytrax, AirlineRatings.com, J.D. Power, and Cirium are separate instruments with different scopes, different inputs, and different definitions of a “good” airline. Disagreement between them is the predictable outcome, not evidence of bias or incompetence.

There is a real problem inside that structure, though. Most consumer-facing rankings publish only part of their methodology — enough to describe the instrument, not enough to reproduce the score.

Weighting is often invisible, operational safety data is almost always absent, and regional context rarely gets adjusted. Travelers end up comparing numbers that were never designed to sit side by side.

Skytrax runs two different instruments under one name

Part of the confusion is baked into the brand. Skytrax produces the World Airline Awards from a passenger-voted survey, but its Official World Airline Star Ratings come from something entirely different: a professional audit. Skytrax is explicit that the star ratings have nothing to do with customer reviews; an auditor works through a checklist, not a comment card.

That checklist typically covers 500 to 800 product and service-delivery items, from the carrier’s home-base airport operation to the cabins and aircraft types that actually fly. One body, two variables — which is why even Skytrax’s own outputs can point in different directions.

The transparency gap starts here. Skytrax publishes only a selection of what the audit checks. The full rating agenda goes to registered industry users — airline, airport, and lounge management — and Skytrax attributes that restriction to commercial reasons.

What the public pages don’t include: a numerical weighting formula, the thresholds at which a rating upgrades or downgrades, or enough of the checklist to reproduce a score independently.

Editorial scoring and passenger sentiment are different variables

AirlineRatings.com adds another layer. Its scores rest on editorial judgment applied to measurable product criteria — seat comfort and catering among them — alongside financial stability and fleet-maintenance records. That’s a check-the-box view of what the airline offers, not what a passenger felt on a given Tuesday.

Its safety assessment is deliberately binary. Points toward a star rating accumulate only when a carrier holds specific certifications or meets defined criteria — and a carrier either has them or it doesn’t.

The inputs include accident history, regulatory compliance status, and holding an IATA Operational Safety Audit certification, commonly called IOSA.

J.D. Power measures something else again: satisfaction. Its 2026 North America Airline Satisfaction Study followed the journey from booking through baggage retrieval, drawing on responses from 10,914 passengers who flew recently. Seven dimensions shape the results — trust, value for price paid, onboard experience, ease of travel, airline staff, digital tools, and pre- and post-flight experience — with respondents split by cabin and traveler type, according to the J.D. Power study.

J.D. Power documents its scope and those seven dimensions publicly. What it does not publish is a weighting formula an outsider could reproduce — and, crucially for this article, it produces no Asia-Pacific ranking comparable to the North American one.

Punctuality is a stopwatch, not a review

Operational analytics live in still another universe. Cirium tracks on-time performance: the share of flights that reach the gate within 14 minutes and 59 seconds of schedule. For its 2025 annual review, Cirium drew on more than 600 real-time flight-information sources.

So the Asia-Pacific punctuality table settles exactly one question: how often a flight reaches its gate on time. It says nothing about cabin product, catering, or whether the airline’s staff smiled during boarding. The narrowness is the point.

ANA finished third for punctuality at 78.88 percent, while Skytrax’s passenger poll placed it fifth worldwide. Singapore Airlines shows the reverse — second on the Skytrax list, fourth at the gate.

One figure is a stopwatch reading; the other is a traveler’s impression of the same journey. Expect those to diverge, and the gap between the two lists stops looking like a mistake.

What none of them measure — and why you can’t reproduce the score

Runway-incursion rates and airport fire-and-rescue response times do not appear as ranking variables in any consumer-facing system reviewed here. Cirium’s punctuality number, built from flight-information feeds, says nothing about how fast an airport’s emergency units would actually respond. No gathered source shows ICAO, the FAA, or a national aviation authority publishing both measures as part of a carrier-ranking dataset.

Disclosure depth also varies sharply. Cirium publishes how many data sources feed its review and where it draws the line on a late arrival. J.D. Power publishes its sample size, its dimensions, and how it splits respondents.

Skytrax describes what its audit covers but keeps the full checklist behind an industry-only login. None of the four publishes enough of its scoring model for an outsider to rebuild the headline rating — which is the gap that makes comparing rankings side by side so difficult.

Regional adjustment is the quietest gap. Skytrax treats its star ratings as a single global standard rather than a set of regional ones, and J.D. Power’s study never leaves North America; neither sets out how Asia-Pacific results should be adjusted for different regulators, local service expectations, or harsher operating environments. It is a question this article can raise but not settle.

How to match the ranking to your actual question

Start with the question, then pick the instrument.

If you’re choosing a long-haul carrier for cabin product and service consistency, the audit-based systems — Skytrax’s star ratings or AirlineRatings.com’s editorial scoring — are built for that question. If safety certification and accident record matter more to you, the checklist-based systems are the ones that score them, criterion by criterion.

If the real worry is a tight connection or a status-qualifying itinerary, only the operational analytics answer it. A punctuality table will not tell you how the cabin felt, and a passenger poll will not tell you whether the flight lands on time.

This also has booking value. An airline can rank poorly on a product audit but rank well on operational reliability. For a traveler protecting a connection, the second number matters more.

Read every ranking as a tool with a defined scope — not as a verdict on “the best airline.”

Key terms

IOSA

IOSA is the IATA Operational Safety Audit, a certification program administered by the International Air Transport Association. IATA is the airline industry’s trade body, and a carrier either holds the certification or it does not. That pass-or-fail shape is what separates a safety input from a service score: certification can be confirmed as present or absent, while cabin experience has to be graded on a scale.

Runway incursion

A runway incursion is any unauthorized or unintended presence on a runway — an aircraft, a vehicle, or a person. Incursions count as ground-safety events, tracked separately from anything that happens in the air. Where a punctuality figure records how a flight ended, an incursion rate would record how close the airport surface came to a serious event, and none of the rankings examined here measures it.

ICAO

ICAO is the International Civil Aviation Organization, the body that coordinates international standards for civil aviation. National regulators such as the FAA translate those standards into the rules carriers actually fly under. Its standards are the reason a worldwide ranking is possible in principle — and the reason a table that ignores how regulators differ leaves part of the story untold.

FAA

The FAA is the Federal Aviation Administration, the national authority that regulates civil aviation in the United States. It writes and enforces the operating rules US carriers and aircraft must follow, and other countries maintain equivalent bodies. In this comparison it stands for the class of regulators whose safety data the consumer rankings leave out.

Questions? Answers.