urlvet/urlvet

<div align="center">

url.vet

sketchy link? just url.vet it

Website: [url.vet](https://url.vet)

Open-source phishing detection engine for URLs and domains — paste any link and get a trust score, a fully explainable verdict, and a shareable security report with live page preview, all in real time. No black-box ML result. Every signal explained.

21 concurrent analyzers · 39 signals · DNS · TLS · domain info · typosquatting · redirect chains · short-link following · content analysis · threat feeds · REST API · Chrome extension · Docker · self-hostable

Open-source self-hostable alternative to VirusTotal, CheckPhish, and URLScan.io

    

English | 中文

⚡ Quick Start · ⚙️ Detection Engine · 🏛 Architecture · 📚 Docs · 🤝 Contributing

_{Also known as URLvet · Previously known as SafeSurf}

</div>

---

Phishing Detection Demo

Paste a URL → get a trust score, verdict, and detailed report in real time.

Image: Phishing Detection Demo

Try it at url.vet, free and with no signup.

Quick Start

git clone https://github.com/urlvet/urlvet.git
cd urlvet
make start

Open Web UI: [localhost:3000](http://localhost:3000)

Detailed setup guide: docs/setup.md

At a Glance

• Live scan, instant results
• 21 analyzers, 39 signals, fully explainable
• HTTP API + Web UI + Chrome extension
• Explainable scoring (no black-box ML)
• Simple Docker setup

How It Compares

| Feature | url.vet | VirusTotal | Google Safe Browsing | URLScan.io | CheckPhish | |---------|----------|------------|----------------------|------------|------------| | Live crawl, instant results | ✅ | Partial | ❌ | Partial | Partial | | Explains every verdict | ✅ | Partial | ❌ | Partial | Partial | | Beginner-friendly interface | ✅ | Partial | Partial | Partial | Partial | | Credential form detection | ✅ | ❌ | ❌ | Partial | ✅ | | Follows redirect chains | ✅ | ✅ | ❌ | ✅ | ✅ | | Detailed technical insights | ✅ | ❌ | ❌ | ✅ | Partial | | Live page preview | ✅ | ❌ | ❌ | ✅ | ✅ | | Detection using AI/ML | ❌ | ✅ | ✅ | Partial | ✅ | | Known phishing database coverage | ✅¹ | ✅ | ✅ | Partial | Partial | | Scan multiple URLs at once | ❌ | ✅ | ✅ | ✅ | ✅ | | Browser protection | ✅ | ✅ | ✅ | ✅ | ❌ | | Open source | ✅ | ❌ | ❌ | ❌ | ❌ |

¹ Checks PhishTank, URLhaus and Google Safe Browsing. On a self-hosted copy, URLhaus and Safe Browsing need free API keys.

Fast scanners (like Google Safe Browsing) give you a verdict from database lookup with no explanation or live scanning. Deep crawlers (like URLScan.io) take too long. url.vet bridges the gap by doing live analysis with per-signal explanations in real time — and it's open-source.

Who This Is For

• End users checking suspicious links
• Privacy-conscious people: no accounts, cookies or trackers, and no IPs or scanned links in logs. The sites you check see our server, never you
• Developers integrating URL analysis
• Security teams building detection pipelines
• Researchers

Vetty

Vetty is the paperclip in the corner of the website. He helps people use url.vet. He is inspired by Clippy, the paperclip helper from old office software.

API Example

Analyze a URL via HTTP:

curl "http://localhost:8080/api/v1/analyze?url=https://example.com"

Sample Response: <pre><code class="language-json"> { "url": "https://example.com", "trust_score": 100, "verdict": "Safe", "reasons": { "good_reasons": [...] } } </code></pre> Full response schema → docs/api.md#example

Detection Engine

21 concurrent goroutines run across 7 signal categories, producing 39 individual signals. Every check emits a reason string — good, bad, or neutral — so the final score is always fully explainable. No black-box verdicts.

Score formula: finalScore = clamp(50 + (trustScore − riskScore) × 0.5) → Risky < 30 · Suspicious 30–64 · Safe ≥ 65

Short links are followed first (through any further short links) and the page they lead to is what gets scanned.

50 is the neutral baseline — a URL with no signals scores exactly 50 (Suspicious), the right default for an unknown URL. Trust signals pull the score up, risk signals pull it down, each weighted at 0.5× so neither dominates alone. Both scores are individually clamped to 0–100 before the formula runs, preventing a single catastrophic signal from drowning all other context.

URL Signals _(8 checks)_

1. Raw IP address as hostname _(common evasion tactic)_
2. Punycode / IDN encoding _(lookalike domain spoofing)_
3. URL shortener _(followed to its destination; one that can't be followed is capped at Suspicious)_
4. Excessive URL length _(over 150 characters; not counted on well-known sites)_
5. Excessive URL path depth _(over 6 path segments; not counted on well-known sites)_
6. Phishing keywords in URL path _(login, verify, secure, update…)_
7. Excessive subdomain count
8. Non-ASCII Unicode characters in hostname _(IDN homograph attack, e.g. аpple.com with Cyrillic а)_

HTTP / Network _(4 checks, single HTTP request)_

1. Redirect chain hop count _(only when the chain leaves the site)_
2. Cross-domain redirect _(final destination differs from source domain, unless it's the same brand)_
3. HSTS support _(read from the scanned site itself, not the last hop)_
4. HTTP status code

DNS _(3 checks)_

1. NS record validity
2. MX record validity
3. IP resolution

TLS / SSL _(2 checks, single TLS handshake)_

1. TLS presence, and a certificate issued for a different hostname
2. Certificate chain — validity, expiry, issuer, CT log status, known-bad fingerprints

Domain Intelligence _(6 checks)_

1. Domain rank _(position in top-1M global popularity list)_
2. TLD trust / risk / ICANN status _(subdomains on hosting services and site builders are judged on their own, not on the host's reputation)_
3. Domain age via RDAP/WHOIS _(under 90 days = risk; over 1, 3 and 5 years = growing trust)_
4. DNSSEC _(cryptographic DNS response integrity)_
5. Shannon entropy score _(flags algorithmically generated domains)_
6. Typosquatting _(5,000 most-visited sites)_ & combo-squatting _(514 curated brands)_

Content Analysis _(12 checks)_

1. Login form on unranked or newly registered domain
2. Payment form _(credit card, CVV fields)_
3. Personal information form
4. Hidden <iframe> _(credential theft / clickjacking vector)_
5. Tracking pixels _(1×1 hidden images)_
6. Brand claimed in the page title vs. hosting domain _(514 brands; lookalike letters folded)_
7. Form submitting to an external domain
8. Password field over unencrypted HTTP
9. Script or meta-refresh redirect _(to a raw IP or another site)_
10. Host or CDN warning page _(Cloudflare's phishing interstitial, 451 takedowns)_
11. Page inside a WordPress system folder _(phishing kits on hacked sites)_
12. Program download _(capped at Suspicious on hosts anyone can upload to)_

Threat Intelligence _(4 checks)_

1. Locally held threat lists _(PhishTank by default; URLhaus, OpenPhish opt-in)_
2. Other pages on the same single-owner host listed
3. Google Safe Browsing / Web Risk _(opt-in; Safe Browsing sees only hash prefixes)_
4. PhishTank API _(only while its list isn't loaded)_

Image: url.vet Analyzer Pipeline

Limitations

• Heuristic-based detection may produce false positives
• No ML model (intentional, prioritizes explainability and auditability)

Not a safety guarantee. Use alongside other defenses.

Architecture

Four containerized services on a shared Docker bridge network. The Go backend is the only service that makes outbound calls to external APIs — the frontend, Chrome, and cache are strictly internal.

Image: url.vet Architecture

| Service | Role | |---|---| | urlvet-web | SvelteKit UI — :3000 (prod) · :5173 (dev) | | urlvet-backend | Go REST API & analyzer engine — :8080 | | urlvet-chrome | Headless Chrome — WebSocket :9222 | | urlvet-valkey | Valkey (Redis-compatible) — :6379, LRU cache, volume-persisted |

Request lifecycle

1. URL submitted via the UI or REST API
2. Backend validates and normalizes the URL (scheme inferred if missing)
3. Valkey cache checked — a hit returns the full result immediately, no re-analysis
4. Short links are followed to their destination, which is scanned instead
5. On miss: 21 goroutines launch concurrently via sync.WaitGroup; panics are recovered per-task without failing the request
6. Results collected → score aggregated → verdict assigned
7. Complete result cached in Valkey (24 h TTL) and logged to scan history
8. Response returned — trust score, verdict, per-signal reasons, redirect chain, page screenshot, per-task timings

server/
  cmd/urlvet/         entry point
  internal/analyzer/    goroutine runner, task definitions, score aggregation
  internal/service/
    checks/             individual analyzer implementations
    screenshot/         headless Chrome integration
    cache/              Valkey client
    threatfeeds/        threat lists, PhishTank, Safe Browsing, Web Risk
    typosquat/          brand similarity engine
web/website/            SvelteKit UI
web/chrome-extension/   browser extension
docker/                 dev & prod Compose configs
docs/                   API, setup, architecture, security

Documentation

| | | |---|---| | Setup | Local & Docker setup, Makefile commands | | Configuration | All environment variables | | Deployment | VPS, reverse proxy, firewall | | API Reference | Endpoints, rate limits, example response | | Architecture | Services, request lifecycle, detection engine | | Security | Admin auth, password hashing | | Performance | Latency, resource usage, tuning | | Design Decisions | Why things are built the way they are | | Maintenance | Cache, logs, backups | | Glossary | Terms and acronyms |

Interactive API docs (Swagger UI): api.url.vet/swagger/index.html

Citation

If you use this project in academic or research work, please cite it — see CITATION.cff.

License

Copyright (C) 2021–2026 Abhishek K P

url.vet is dual-licensed:

• Community — GNU Affero General Public License v3.0. Free to use, modify, and self-host. Any modified version run over a network must make its source code available to users.
• Commercial — A separate commercial license is available for organizations that cannot comply with the AGPL-3.0 (e.g. closed-source SaaS).

Contributing

• Found a bug? → Open an issue
• Have a question or idea? → Start a discussion
• Want to contribute code? → CONTRIBUTING.md

If you found this project helpful, consider giving it a star.

<!-- <div align="center"> <a href="https://star-history.com/#urlvet/urlvet&Date"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=urlvet/urlvet&type=Date&theme=dark" /> <source