Import pipeline tests against a small fixture corpus #20

New issue

Open

opened 2026-09-26 13:58:34 +00:00 by belvedere · 0 comments

belvedere commented

2026-09-26 13:58:34 +00:00

Collaborator

Objective: The importer's parsing and idempotency are covered without network access.

Files:

Create: test/fixtures/frequency/en_50k.txt (a few dozen lines, including noise tokens)
Create: test/first_thousand_words/import/frequency_test.exs
Create: test/first_thousand_words/import/concepts_test.exs

Steps:

Assert noise tokens ('s, --, 123) are rejected.
Assert the same word appearing twice upserts to one row.
Assert running the concept import twice produces identical concept counts.
Stub the HTTP download; tests must never hit the network.

Verify: mix test test/first_thousand_words/import/ — all pass, offline.

**Objective:** The importer's parsing and idempotency are covered without network access. **Files:** - Create: `test/fixtures/frequency/en_50k.txt` (a few dozen lines, including noise tokens) - Create: `test/first_thousand_words/import/frequency_test.exs` - Create: `test/first_thousand_words/import/concepts_test.exs` **Steps:** 1. Assert noise tokens (`'s`, `--`, `123`) are rejected. 2. Assert the same word appearing twice upserts to one row. 3. Assert running the concept import twice produces identical concept counts. 4. Stub the HTTP download; tests must never hit the network. **Verify:** `mix test test/first_thousand_words/import/` — all pass, offline.

belvedere added this to the M2 — Lexicon model & data pipeline milestone

2026-09-26 13:58:34 +00:00

belvedere added the

labels

2026-09-26 13:58:34 +00:00

belvedere referenced this issue

2026-09-26 14:02:04 +00:00

Import pipeline tests against a small fixture corpus #72