Methodology

What we investigate

We examine products across seven evidence layers. A contradiction is found when what a company says in one layer doesn't match what's observed in another. Different product types use different layers:

How findings are produced

1. Product selection. We prioritise products people actually use, products with known controversies, and products where public perception doesn't match evidence (e.g. Apple “privacy-first”).
2. Evidence gathering. We collect publicly available evidence: privacy policies, app store listings, Exodus Privacy tracker reports, FCC filings, court documents, academic studies, news investigations, and (where possible) network traffic captures from physical devices.
3. AI-assisted analysis. Claude identifies potential contradictions between evidence layers. This is where AI provides scale — it can cross-reference a privacy policy against 50 news articles faster than a human researcher.
4. Human review. Every contradiction is reviewed by a human. We check: Is the claim accurate? Is the evidence real and correctly attributed? Does the plain-language summary fairly represent both sides? Is the severity rating justified?
5. Source citation. We are actively adding verifiable source links to every contradiction. Each source should allow a reader to click through and verify the finding independently within 30 seconds.
6. Publication. Findings are stored in a structured database, then rendered into the static site you're reading. The site itself uses zero external dependencies, no tracking, no analytics, no cookies.

Source citation progress

We are working to add direct source URLs to every finding. On each device page, findings with sources show clickable links in the expanded details. Our goal is 100% source coverage before public launch.

Severity ratings

Each contradiction is rated by severity based on real-world impact:

Grading

Products are graded A+ through F based on average contradiction severity. Software products use stricter thresholds than hardware because software has more publicly available research (more evidence means more contradictions for even average products). The grade reflects what we found, not an overall product quality judgement. A product with fewer publicly documented issues may simply have less available evidence.

What we get wrong

We make mistakes. AI-assisted research at scale means some findings may be imprecise, out of date, or missing context. Common failure modes:

• Stale information. A company may have changed practices since the evidence was gathered. We try to date our sources but some findings may not reflect the current state.
• Missing context. A finding may be technically accurate but miss an important nuance — for example, a feature that was briefly available in one region but not others.
• Severity disputes. Reasonable people can disagree about whether something is “critical” or “high.” Our ratings reflect our assessment of real-world harm.
• Attribution. Some practices are industry-wide but we attribute them to specific products. We try to note when something is standard practice vs uniquely bad.

Corporate inherited risks

Some risks belong to a parent company, not an individual product. An FTC consent decree, PRISM membership, or China's National Intelligence Law applies to the corporate entity and every product it owns. These are structural risks — legal obligations and corporate practices that individual product settings cannot override.

Why this matters: WhatsApp's end-to-end encryption does not exempt it from Meta's FTC consent decree. TikTok's "Project Texas" (US data storage) does not exempt ByteDance from China's Intelligence Law. When Apple removed Advanced Data Protection in the UK, every Apple product in the country lost encrypted backups — individual products had no say.

Grade floor: Inherited risks set a minimum grade for all products under that entity. If a parent company's structural risks compute to a D, no product under that parent can score better than D, regardless of its own privacy settings. The product's own contradictions can make its grade worse, but never better than the parent's floor. When a product's grade is raised by its parent's floor, this is shown on the product page.

Dynamic: Entity grades and floors recalculate automatically when new risks are added. If a parent company receives a new regulatory action or its jurisdiction changes, every linked product's grade updates on the next site generation. View an example: Google (Alphabet) →

Independence

DeviceGuardian is an independent research project. We have no commercial relationships with any product we investigate. We don't accept payment for reviews, and we don't modify findings based on company requests. Our REPLACE recommendations (suggesting alternatives to poorly-graded products) are based on evidence, not partnerships. The site itself collects zero user data — no analytics, no cookies, no tracking pixels. We practice what we preach.