Technical overviewData sourcing and workflow automation

JobSpy Search

A repeatable sourcing pipeline that turns fragmented job boards into review-ready data.

Role
Solo developer across scraping orchestration, source adapters, pandas transformations, screening, and contact matching
Evidence status
Used in the active job-search workflow; outputs verified across two sourcing batches and one referral scan
Verified
2026-07-23

01

Purpose and scope

JobSpy Search automates broad job discovery across Indeed, LinkedIn, Google Jobs, and a custom CareerBeacon adapter. It normalizes and deduplicates results, applies explicit screening rules, produces compact review files, and matches hiring companies against a LinkedIn connections export for warm-referral opportunities.

What it is used for

  • Run the same role and location search matrix without reopening every job board manually.
  • Separate deterministic exclusions from the human judgment required for fit and red flags.
  • Find warm contacts at companies with current matching openings.
  • Python
  • JobSpy
  • pandas
  • HTML parsing
  • CSV
  • Automation

02

Architecture

Configuration and inputs

Configuration

Search matrix

Roles, locations, sites, age limits, and result caps

Input data

LinkedIn connections CSV

Company and relationship data for referral matching

Parallel queries

Acquisition layer

External adapters

JobSpy providers

Indeed, LinkedIn, and Google provider interface

Custom adapter

CareerBeacon adapter

HTML parsing with province and title post-filters

Shared row schema

Processing pipeline

pandas

Normalize and dedupe

Concatenates sources and removes repeated postings

Deterministic filter

Rule-based screener

Applies explicit standing exclusions

Referral analysis

Company matcher

Exact, alias, and reviewed fuzzy matching

Writes

Review outputs

Archive

Full results CSV

All columns and job descriptions

Triage

Lean screening CSV

Compact fields for first-pass fit review

Human review

Referral matches

Warm contacts grouped by hiring company

Text equivalent: A configurable role-and-location matrix fans out to JobSpy providers and the custom CareerBeacon adapter. All sources return a shared row shape that pandas normalizes and deduplicates. Deterministic screening and company matching then produce full, lean, and warm-referral CSV outputs for human review.

03

Engineering decisions

Adapter boundary for unsupported boards

Boards outside JobSpy's fixed provider list implement the same row shape independently, so CareerBeacon can join the pipeline without forking the library.

Full and lean review files

The pipeline preserves full descriptions for shortlisted roles while generating a much smaller column set for fast first-pass triage.

Automation stops before judgment

Code handles sourcing, deduplication, and standing exclusions; fit assessment, red flags, and application decisions remain explicit human steps.

04

Verification evidence

  • The first production run produced 414 deduplicated postings and 266 rows after deterministic screening.
  • A second batch produced 434 deduplicated postings and 283 screened rows before human fit review.
  • The first connection scan identified 286 exact company matches, two reviewed aliases, and 12 retained fuzzy matches.
  • CareerBeacon and the connection matcher were merged through reviewed pull requests and used in the live workflow.
verified-2026-07-23
$ python search_jobs.py
sources: Indeed + LinkedIn + Google + CareerBeacon
434 deduplicated -> 283 screened
$ python match_connections.py
✓ exact + alias + reviewed fuzzy matches written

05

Limitations

Next step

Discuss the engineering behind this work.

Email meDownload résuméRepository remains private