Commit Graph

11 Commits

Author SHA1 Message Date
soroush.asadi 6b657c7795 Applicants: auto-tags + deep search w/ highlight; never delete (archive instead)
CI/CD / CI · dotnet build (push) Successful in 2m1s
CI/CD / Deploy · hamkadr (push) Successful in 2m36s
- Tags: parser extracts cert/skill keywords (mmt, ICU/CCU, دیالیز, اتاق عمل,
  اورژانس, مسئول فنی, پروانه‌دار…) + role + city into TalentListing.Tags
  (+ migration); shown as chips on cards.
- Deep search on /Talent: «جستجوی عمیق» box does Postgres ILIKE across
  tags, description, person, area, role, city (every term must match);
  matches are highlighted with <mark> via SearchHighlight.
- Never delete: ShiftStatus.Archived + the admin «بایگانی گروهی» action now
  ARCHIVES aggregated posts (hidden from site, kept in DB) and leaves the
  raw crawl rows intact — a permanent archive for future analytics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 11:25:32 +03:30
soroush.asadi e4dc5180ad Applicants: 1→N contact methods with types (phone/email/Instagram/Telegram/Bale/site)
CI/CD / CI · dotnet build (push) Successful in 1m32s
CI/CD / Deploy · hamkadr (push) Successful in 1m31s
- ContactMethod entity (Type + Value + SortOrder) 1→N on TalentListing (+ migration).
- Parser extracts ALL contacts: multiple phones + landlines, email, and
  socials (Instagram/Telegram/Bale/WhatsApp/website) via URLs and Persian
  keyword cues; primary Phone kept for cards.
- ContactInfo helper: per-type label/icon/clickable href (tel:/mailto:/t.me/…).
- Ingestion attaches contacts to each (fanned-out) talent listing; manual
  Review re-parses to attach them + the admin-typed phone.
- Talent details renders the full contact list as buttons; falls back to the
  single phone, then the Divar source link.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 11:10:19 +03:30
soroush.asadi 48760c4e83 Multi-role ads: parse all roles + fan-out publish one listing per role
CI/CD / CI · dotnet build (push) Successful in 2m16s
CI/CD / Deploy · hamkadr (push) Has been cancelled
An ad like «استخدام پرستار سالمند و کودک و همراه بیمار» names several roles;
we kept only the first. Now:
- Parser collects ALL roles (ParsedListing.RoleNames): exact taxonomy
  matches (substring-deduped so پرستار⊂پرستار سالمندان) plus synonyms
  (سالمند→پرستار سالمندان, کودک/همراه بیمار→پرستار, اتاق عمل→تکنسین اتاق عمل…),
  capped at 4.
- Ingestion publishes one Shift/Job/Talent per resolved role (AI role +
  parser roles, distinct, capped), so each role is independently
  browsable and filterable. RawListing dedupe is unchanged (one raw → N posts).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 10:58:29 +03:30
soroush.asadi 2bb8771ade Normalize ریال→تومان pricing; stop exposing crawl source (medjobs/telegram)
CI/CD / CI · dotnet build (push) Successful in 29s
CI/CD / Deploy · hamkadr (push) Successful in 42s
- Parser now reads the currency: ریال amounts (incl. «میلیون ریال» and
  numbers with no تومان unit but ≥200M) are converted to تومان (÷10), so
  «۴۰۰٬۰۰۰٬۰۰۰ ریال» shows as ۴۰٬۰۰۰٬۰۰۰ تومان instead of 400M.
- Aggregated facility fallback name no longer embeds the source
  («مرکز درمانی (از مدجابز)» → «مرکز درمانی (نامشخص)»).
- Talent details only ever names Divar as a fallback source (when the
  number couldn't be extracted); medjobs/telegram are never shown publicly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 09:05:34 +03:30
soroush.asadi 213af9db48 AI tag/category assignment + phone extraction from web ads
CI/CD / CI · dotnet build (push) Successful in 2m37s
CI/CD / Deploy · hamkadr (push) Successful in 1m11s
AI (when enabled, now that the server proxy is up):
- AiStructured gains phone, personName, yearsExperience, isLicensed.
- The auditor appends an authoritative output-schema to the admin prompt
  so classification stays correct even with an older stored prompt — it
  now classifies kind as shift|job|talent and extracts the contact phone
  and talent details.
- Ingestion publish prefers the AI's tags (kind/role/city/facility/phone +
  talent fields) over the heuristic parser when present.
- Default prompt updated to describe the three kinds + new fields.

Phone extraction from websites (Medjobs / generic sites), where the
number sits behind a "تماس با این آگهی" reveal:
- HtmlUtil.HarvestPhones scans the full markup for tel: links, JSON-LD
  "telephone", data-*phone* attributes, and inline Iranian mobile/landline
  numbers (Persian digits folded), normalized (mobiles 09…, landlines 0…).
- Medjobs + Website sources append harvested numbers to the ad text so the
  parser/AI capture them; manual review then prefills the phone too.
- Parser phone extraction now also captures a landline as a fallback.

Note: if a site loads the number purely via XHR (not in HTML), a
per-source reveal endpoint would be a follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 08:11:14 +03:30
soroush.asadi 4e5df73cf7 Add «آماده به کار» (talent) listing type — workers offering themselves
CI/CD / CI · dotnet build (push) Successful in 1m41s
CI/CD / Deploy · hamkadr (push) Has been cancelled
Adds a third listing kind alongside Shift/Job for healthcare staff who
advertise their own availability (very common in Iranian medical
channels, e.g. "دندانپزشک آماده همکاری… ۵۰٪ تسویه"). These have no
facility; the contact phone is the key field.

- Model: TalentListing (role, person name, years, licensed, city/district,
  area note, availability, gender, comp, phone) + ListingKind.Talent +
  RawListing.LinkedTalentId + DbSet/relations/indexes + EF migration.
- Parser: detect آماده‌به‌کار/جویای کار → Kind=Talent; extract person name,
  years of experience, licensed flag, area («منطقه ۱»), phone. Facility
  name extraction now skipped for talent.
- Validator: talent path scores role + phone + medical (no facility/pay
  required).
- Ingestion auto-publish: creates a TalentListing for talent kind.
- Review (manual publish): Talent option + talent fields; publishes a
  TalentListing without a facility. Shift/Job facility now falls back to a
  shared «نامشخص / ثبت نشده» record when the ad names none — publishing
  never fails on a missing facility.
- Browse /Talent (indexable, filters: city/district/role/gender),
  details /Talent/Details (noindex — personal contact, tel: call button),
  _TalentCard, badge-talent, nav link, home section.
- Sitemap includes /Talent; robots disallows /Talent/Details. Archiver
  expires stale talent listings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 08:01:12 +03:30
soroush.asadi e6a796ab27 Match crawled listings to existing facilities (fuzzy) before creating new
CI/CD / CI · dotnet build (push) Successful in 1m28s
CI/CD / Deploy · hamkadr (push) Successful in 2m24s
When publishing a scraped listing we now look for a facility we already
have that is exactly or closely the same, and only create a new one when
there is no match — avoiding duplicates like «بیمارستان میلاد» vs «میلاد».

- ListingParser: extract a facility name (keyword + distinctive words) from
  the post and surface it in the parser notes.
- FacilityMatcher: Persian-aware normalization (ي/ك, ZWNJ, punctuation),
  type-word stripping for a "core" name, contains + Levenshtein similarity,
  and FindBest (same-city exact → any-city exact → same-city fuzzy → fuzzy).
- Review (manual publish): auto-select a matching facility or prefill the
  new-facility name; resolve-or-create uses fuzzy match; dropdown preselects.
- IngestionService (auto-publish): reuse FacilityMatcher against a run-wide
  facility list (grows as new ones are created) instead of exact-name only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 07:14:48 +03:30
soroush.asadi 487c7ca82f [Ingest] Persistent crawl run-log + per-source breakdown on admin queue
CI/CD / CI · dotnet build (push) Has been cancelled
CI/CD / Deploy · hamkadr (push) Has been cancelled
Each ingestion run now records an IngestionRun row (found/queued/published/flagged/spam/duplicates + a per-source detail string). Admin → صف آگهی‌ها shows a «تاریخچه جمع‌آوری» table of the last 15 runs (hover a row for the per-source breakdown), so admins can see how much each source found vs added over time. IngestionSummary gains TotalFetched. Migration: IngestionRuns table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 06:23:58 +03:30
soroush.asadi 3c08c1a265 Move ingestion + Telegram/Bale/Divar config to DB-backed admin settings
CI/CD / CI · dotnet build (push) Successful in 6m22s
CI/CD / Deploy · hamkadr (push) Failing after 3s
- AppSetting gains source config: AutoIngestEnabled, IngestIntervalMinutes, Telegram/Bale/Divar enabled+channels/token/queries
- IListingSource.FetchAsync(AppSetting) — sources read config from DB, not IOptions/appsettings; sample source dev-only
- IngestionWorker reads AutoIngest+interval from DB each cycle (toggle at runtime, no redeploy)
- /Admin/Settings gets a 'منابع جمع‌آوری' section; removed Ingestion env/appsettings + compose env vars
- ENV_FILE shrinks to HOST_PORT + POSTGRES_* + ADMIN_PHONE (AI + sources are all in-admin); migration

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 00:44:11 +03:30
soroush.asadi 36bb165438 Real channel fetch (Telegram/Bale/Divar) + AI-audited automation engine + CI/CD
- Fetch: Telegram via t.me/s, Bale via Bot API, Divar via web-search (HttpClient, config-gated, graceful)
- AI layer: DB-backed AppSetting (mode auto/manual, thresholds, AI endpoint/model/key/prompt/framework, auto-approve); OpenAI-compatible IAiAuditor (self-host/Iranian endpoints; fails safe to manual)
- Pipeline: fetch → dedupe(hash) → parse → validate → AI audit → Discard/Flag/Queue/auto-publish (resolve-or-create facility)
- Admin: /Admin/Settings automation+AI panel; queue shows confidence + AI verdict; flagged section
- CI/CD: Dockerfile, docker-compose.prod.yml, .gitea/workflows/ci-cd.yml, nginx vhost, DEPLOY.md; forwarded headers + /healthz + prod reference-only seed; ports 22/80/443 only

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 17:41:02 +03:30
soroush.asadi 931b7b6ffb Add scrape/ingestion engine + validation, and 24h shift hour-range visualization
Scrape engine (Services/Scraping/): pluggable IListingSource (working sample + Telegram/Divar credential-ready stubs) → IngestionService (content-hash dedupe → parse → validate → review queue) → ListingValidator (completeness score + spam screen) → IngestionWorker (config-gated hosted service). RawListing gains ContentHash/Confidence/ValidationNotes; RawListingStatus.Flagged. Admin /Admin gets run-now, source list, confidence + flagged queue.

Hour-range viz: _HourBar 24h timeline bar (colored by type, overnight wrap) on shift cards, recommendation cards, and detail.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 08:18:19 +03:30