soroushdes/hamkadr
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add scrape/ingestion engine + validation, and 24h shift hour-range visualization

Browse Source 
Scrape engine (Services/Scraping/): pluggable IListingSource (working sample + Telegram/Divar credential-ready stubs) → IngestionService (content-hash dedupe → parse → validate → review queue) → ListingValidator (completeness score + spam screen) → IngestionWorker (config-gated hosted service). RawListing gains ContentHash/Confidence/ValidationNotes; RawListingStatus.Flagged. Admin /Admin gets run-now, source list, confidence + flagged queue.

Hour-range viz: _HourBar 24h timeline bar (colored by type, overnight wrap) on shift cards, recommendation cards, and detail.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
soroush.asadi
2026-06-03 08:18:19 +03:30
parent 69fa921fbd
commit 931b7b6ffb
24 changed files with 1439 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/JobsMedical.Web/Program.cs
+17
View File
@@ -21,6 +21,23 @@ builder.Services.AddScoped<OtpService>();
// Listing parser: heuristic now; swap for an LLM-backed IListingParser later.
builder.Services.AddSingleton<IListingParser, HeuristicListingParser>();
// Scrape/ingestion engine: pluggable sources → dedupe → parse → validate → review queue.
builder.Services.Configure<JobsMedical.Web.Services.Scraping.IngestionOptions>(
builder.Configuration.GetSection("Ingestion"));
builder.Services.Configure<JobsMedical.Web.Services.Scraping.TelegramOptions>(
builder.Configuration.GetSection("Ingestion:Telegram"));
builder.Services.Configure<JobsMedical.Web.Services.Scraping.DivarOptions>(
builder.Configuration.GetSection("Ingestion:Divar"));
builder.Services.AddSingleton<JobsMedical.Web.Services.Scraping.ListingValidator>();
builder.Services.AddSingleton<JobsMedical.Web.Services.Scraping.IListingSource,
JobsMedical.Web.Services.Scraping.SampleListingSource>();
builder.Services.AddSingleton<JobsMedical.Web.Services.Scraping.IListingSource,
JobsMedical.Web.Services.Scraping.TelegramListingSource>();
builder.Services.AddSingleton<JobsMedical.Web.Services.Scraping.IListingSource,
JobsMedical.Web.Services.Scraping.DivarListingSource>();
builder.Services.AddScoped<JobsMedical.Web.Services.Scraping.IngestionService>();
builder.Services.AddHostedService<JobsMedical.Web.Services.Scraping.IngestionWorker>();
// Phone-OTP cookie auth.
builder.Services.AddAuthentication(CookieAuthenticationDefaults.AuthenticationScheme)
.AddCookie(o =>