Import pipeline tests against a small fixture corpus #72

New issue

Closed

opened 2026-09-26 13:58:59 +00:00 by belvedere · 1 comment

belvedere commented

2026-09-26 13:58:59 +00:00

Collaborator

Objective: The importer's parsing and idempotency are covered without network access.

Files:

Create: test/fixtures/frequency/en_50k.txt (a few dozen lines, including noise tokens)
Create: test/first_thousand_words/import/frequency_test.exs
Create: test/first_thousand_words/import/concepts_test.exs

Steps:

Assert noise tokens ('s, --, 123) are rejected.
Assert the same word appearing twice upserts to one row.
Assert running the concept import twice produces identical concept counts.
Stub the HTTP download; tests must never hit the network.

Verify: mix test test/first_thousand_words/import/ — all pass, offline.

**Objective:** The importer's parsing and idempotency are covered without network access. **Files:** - Create: `test/fixtures/frequency/en_50k.txt` (a few dozen lines, including noise tokens) - Create: `test/first_thousand_words/import/frequency_test.exs` - Create: `test/first_thousand_words/import/concepts_test.exs` **Steps:** 1. Assert noise tokens (`'s`, `--`, `123`) are rejected. 2. Assert the same word appearing twice upserts to one row. 3. Assert running the concept import twice produces identical concept counts. 4. Stub the HTTP download; tests must never hit the network. **Verify:** `mix test test/first_thousand_words/import/` — all pass, offline.

belvedere added this to the M2 — Lexicon model & data pipeline milestone

2026-09-26 13:58:59 +00:00

belvedere added the

labels

2026-09-26 13:58:59 +00:00

belvedere commented

2026-09-26 14:02:04 +00:00

Author

Collaborator

Duplicate of #20 — created by a re-run whose pagination helper only saw the first page of issues. Closing to keep the backlog clean. Repo owner can delete this outright.

belvedere closed this issue

2026-09-26 14:02:04 +00:00