Work sample 15 / 15Websites & scripts
Lead list built from public directories into a checked Excel file
How I collect business listings from public directories, politely and reliably, into a de-duplicated Excel list ready for a CRM.
- Area
- Websites & scripts
- Tools
- Python · Excel
- Typical scope
- US$1,000–2,000
- Timeline
- About 2 weeks · 4 fixed milestones
Work sample: realistic data, built to show how I approach this kind of project. Not a past client job.
01The problem
Building a lead list by copy and paste from directories takes days and the result is full of duplicates and gaps. A script can do it in minutes, but only if it respects the site, handles layout changes and retries errors.
The deliverable is a clean, checked list with a log of every request.
Owners usually notice it as
- Lists built by copy and paste
- Duplicates across categories
- Missing phones or websites
- No record of where each row came from
02Scope
A fixed scope agreed in writing before anything is built. The table shows what a project like this usually covers and what it leaves out.
| Area | Included |
|---|---|
| Sources | Agreed public directories and categories |
| Collection | robots.txt respected, pagination followed, two layouts handled |
| Reliability | Retries and a request log |
| Clean-up | De-duplicated, phones standardised, categories mapped |
| Output | Excel and CSV ready for CRM import |
| Not included | Sites that forbid collection, and personal data |
Typical volumes for a business of this size. The sample files are sized as described in section 05.
| Milestone | Deliverable | Timing | Share | Weeks 1–2 |
|---|---|---|---|---|
| M1 Scope | Sources, fields and rules agreed | Week 1 | 25% | |
| M2 Script | Collection script and log | Week 1 | 35% | |
| M3 Clean | De-duplicated, checked list | Week 2 | 25% | |
| M4 Handover | Files, script, guide | Week 2 | 15% |
Typical scope for this kind of project: US$1,000–2,000
About 2 weeks across 4 fixed milestones. The fixed quote follows a US$300 Systems Health Check, which confirms the volumes and the scope first. Your software subscriptions are billed to you by the vendors.
03How I approach it
Construction drawing. Each stage has its own check before the next one starts.
- 01
Check the rules
robots.txt and terms first.
- 02
Collect gently
Rate-limited requests with retries.
- 03
Handle layout changes
Parsers for each layout, with a test page for each.
- 04
Clean and de-duplicate
Same business across categories becomes one row.
- 05
Log the source
Every row keeps its source URL.
04Checks
Nothing goes live on trust. These checks run before sign-off and stay in the files, so they can be re-run after every change.
- Every listing has a source URL
- No duplicate businesses
- Failed requests retried and logged
- Phones in a standard format
- Row count matches the log
| Check | What it catches | Status |
|---|---|---|
| Duplicate listing IDs | Listed on the Checks tab | PASS |
| Phones not in +61 format (10 digits) | Listed on the Checks tab | PASS |
| Listings without a name | Listed on the Checks tab | PASS |
| Ratings outside 1-5 | Listed on the Checks tab | PASS |
| Emails without @ | Listed on the Checks tab | PASS |
| Listing pages that failed after retries | Listed on the Checks tab | PASS |
| Requests to robots.txt-disallowed paths | Listed on the Checks tab | PASS |
| Unique listings vs category entries minus cross-listings | Listed on the Checks tab | PASS |
Status read from the sample workbook. Each check is a formula, not a typed value.
05Handover
Everything is handed over in editable files you own, with the steps written down so your team is not dependent on me.
In this work sample
A Python collection script run against a test directory: 158 listings from 182 requests, with 8 checks.
- Excel and CSV list
- Script with comments
- Request log
- Guide to re-running it
06Files in this work sample
Screens from the sample files. Select one to open it full size.
Directory-Listings.xlsx › Listings
Directory-Listings.xlsx › Checks
Directory-Listings.xlsx › Summary
| File | What it is | Size |
|---|---|---|
| scrape_directory.py | The scraper: robots.txt, crawl delay, pagination, two page templates, retries, de-duplication | 6.6 KB |
| Directory-Listings.xlsx | Output workbook: listings, summary, checks and request log | 28 KB |
| directory_listings.csv | Output CSV | 30.4 KB |
| scrape_log.csv | Every request with status and timing | 5.4 KB |
Tell me what’s bugging you and I’ll tell you where I’d look first.
Send a short note on Upwork about what is slowing the business down. I’ll reply with where I’d start, and whether the US$300 Systems Health Check is the right first step.