acrius/food-ai
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: improved receipt recognition, batch product add, and scan UX

Browse Source 
- Rewrite receipt OCR prompt: completes truncated names, preserves fat%
  and flavour attributes, extracts weight/volume from line, infers
  typical package sizes for solid goods with quantity_confidence field
- Add quantity_confidence to RecognizedItem, EnrichedItem, and
  ProductJobResultItem; propagate through item enricher and worker
- Replace per-item create loop with single POST /user-products/batch call
  from RecognitionConfirmScreen
- Rebuild RecognitionConfirmScreen: amber qty border for low
  quantity_confidence, tappable product name → catalog picker,
  sort items by confidence, full L10n (no hardcoded strings)
- Add timestamps (HH:mm / d MMM HH:mm) to recent scan chips
- Show close-app hint on ProductJobWatchScreen (queued + processing)
- Refresh recentProductJobsProvider on watch screen init so new job
  appears without a manual pull-to-refresh
- App-level WidgetsBindingObserver refreshes product and dish job lists
  on resume, fixing stale lists after background/foreground transitions
- Add 9 new L10n keys across all 12 locales

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
dbastrikin
2026-03-26 23:09:57 +02:00
parent b2bdcbae6f
commit 5c5ed25e5b
38 changed files with 1221 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
backend/internal/adapters/ai/types.go
View File

@@ -81,11 +81,12 @@ type MealEntry struct {
// RecognizedItem is a food item identified in an image.
type RecognizedItem struct {
Name string `json:"name"`
Quantity float64 `json:"quantity"`
Unit string `json:"unit"`
Category string `json:"category"`
Confidence float64 `json:"confidence"`
Name string `json:"name"`
Quantity float64 `json:"quantity"`
Unit string `json:"unit"`
Category string `json:"category"`
Confidence float64 `json:"confidence"`
QuantityConfidence float64 `json:"quantity_confidence"`
}
// UnrecognizedItem is text from a receipt that could not be identified as food.

36
backend/internal/adapters/openai/recognition.go
View File

@@ -26,21 +26,41 @@ func (c *Client) RecognizeReceipt(ctx context.Context, imageBase64, mimeType, la
prompt := fmt.Sprintf(`You are an OCR system for grocery receipts.
Analyse the receipt photo and extract a list of food products.
For each product determine:
- name: product name (remove article codes, extra symbols)
- quantity: amount (number)
- unit: unit (g, kg, ml, l, pcs, pack)
- category: dairy | meat | produce | bakery | frozen | beverages | other
- confidence: 0.0–1.0
Skip items that are not food (household chemicals, tobacco, alcohol).
Rules for each product:
NAME (confidence):
- Remove article codes, cashier codes (e.g. "1/72", "4607001234"), extra symbols.
- Complete obviously truncated OCR names: "Паштет шпро." → "Паштет шпротный",
"Паштет с говяжьей пече" → "Паштет с говяжьей печенью".
- Preserve meaningful product attributes: fat percentage ("3.2%%", "жирн. 9%%"),
flavour ("с гусиной печенью", "яблочный"), brand qualifiers ("ультрапастеризованное").
- confidence: your certainty that the name is correct (0.0–1.0).
QUANTITY + UNIT (quantity_confidence):
- If a weight or volume is written on the receipt line (e.g. "160г", "1л", "500 мл", "0.5кг"),
use it as quantity+unit. quantity_confidence = 0.9–1.0.
- If the count on the receipt is 1 and no weight/volume is stated, but the product is a
liquid (juice, milk, kefir, etc.) — infer 1 l and set quantity_confidence = 0.5.
- If the count is 1 and no weight is stated, but the product is a solid packaged good
(pâté, spreadable cheese, sausage, butter, hard cheese, etc.) — infer a typical
package weight in grams (e.g. pâté 100 g, spreadable cheese 180 g, butter 200 g)
and set quantity_confidence = 0.35.
- If the receipt explicitly states the quantity and unit (e.g. "2 кг", "3 шт"),
use them directly. quantity_confidence = 1.0.
- Never output quantity = 1 with unit = "g" unless the receipt explicitly says "1 г".
- unit must be one of: g, kg, ml, l, pcs, pack.
CATEGORY: dairy | meat | produce | bakery | frozen | beverages | other
Skip items that are not food (household chemicals, tobacco, alcohol, bags, services).
Items with unreadable text — add to unrecognized.
Return all text fields (name) in %s.
Return ONLY valid JSON without markdown:
{
"items": [
{"name": "...", "quantity": 1, "unit": "l", "category": "dairy", "confidence": 0.95}
{"name": "...", "quantity": 160, "unit": "g", "category": "other", "confidence": 0.95, "quantity_confidence": 0.9}
],
"unrecognized": [
{"raw_text": "...", "price": 89.0}

15
backend/internal/domain/recognition/handler.go
View File

@@ -103,13 +103,14 @@ type imagesRequest struct {
// EnrichedItem is a recognized food item enriched with ingredient_mappings data.
type EnrichedItem struct {
Name string `json:"name"`
Quantity float64 `json:"quantity"`
Unit string `json:"unit"`
Category string `json:"category"`
Confidence float64 `json:"confidence"`
MappingID *string `json:"mapping_id"`
StorageDays int `json:"storage_days"`
Name string `json:"name"`
Quantity float64 `json:"quantity"`
Unit string `json:"unit"`
Category string `json:"category"`
Confidence float64 `json:"confidence"`
QuantityConfidence float64 `json:"quantity_confidence"`
MappingID *string `json:"mapping_id"`
StorageDays int `json:"storage_days"`
}
// ReceiptResponse is the response for POST /ai/recognize-receipt.

13
backend/internal/domain/recognition/item_enricher.go
View File

@@ -26,12 +26,13 @@ func (enricher *itemEnricher) enrich(enrichContext context.Context, items []ai.R
result := make([]EnrichedItem, 0, len(items))
for _, item := range items {
enriched := EnrichedItem{
Name: item.Name,
Quantity: item.Quantity,
Unit: item.Unit,
Category: item.Category,
Confidence: item.Confidence,
StorageDays: 7, // sensible default
Name: item.Name,
Quantity: item.Quantity,
Unit: item.Unit,
Category: item.Category,
Confidence: item.Confidence,
QuantityConfidence: item.QuantityConfidence,
StorageDays: 7, // sensible default
}
catalogProduct, matchError := enricher.productRepo.FuzzyMatch(enrichContext, item.Name)

15
backend/internal/domain/recognition/product_job.go
View File

@@ -20,13 +20,14 @@ type ProductImagePayload struct {
// ProductJobResultItem is an enriched product item stored in the result JSONB.
type ProductJobResultItem struct {
Name string `json:"name"`
Quantity float64 `json:"quantity"`
Unit string `json:"unit"`
Category string `json:"category"`
Confidence float64 `json:"confidence"`
MappingID *string `json:"mapping_id,omitempty"`
StorageDays int `json:"storage_days"`
Name string `json:"name"`
Quantity float64 `json:"quantity"`
Unit string `json:"unit"`
Category string `json:"category"`
Confidence float64 `json:"confidence"`
QuantityConfidence float64 `json:"quantity_confidence"`
MappingID *string `json:"mapping_id,omitempty"`
StorageDays int `json:"storage_days"`
}
// ProductJobResult is the JSONB payload stored in product_recognition_jobs.result.

15
backend/internal/domain/recognition/product_worker.go
View File

@@ -127,13 +127,14 @@ func (pool *ProductWorkerPool) processJob(workerContext context.Context, jobID s
resultItems := make([]ProductJobResultItem, len(enriched))
for index, item := range enriched {
resultItems[index] = ProductJobResultItem{
Name: item.Name,
Quantity: item.Quantity,
Unit: item.Unit,
Category: item.Category,
Confidence: item.Confidence,
MappingID: item.MappingID,
StorageDays: item.StorageDays,
Name: item.Name,
Quantity: item.Quantity,
Unit: item.Unit,
Category: item.Category,
Confidence: item.Confidence,
QuantityConfidence: item.QuantityConfidence,
MappingID: item.MappingID,
StorageDays: item.StorageDays,
}
}