← Back to Portfolio★ Featured Project
Shopify Sync Platform

Sync Agent

The invisible infrastructure that keeps multi-store retail operations in perfect sync. Webhook-driven inventory, pricing, content, and image synchronization across unlimited Shopify stores — with echo-loop prevention, delta computation, and full reconciliation to ensure every storefront reflects the single source of truth in real time.

● Production 💻 Express + BullMQ 📊 Prometheus Metrics 🔒 Private Repository
12
DB Models
37
API Endpoints
16
GraphQL Ops
11K+
Lines of Code
4
Sync Types
5+
Cron Jobs

What Gets Synced

Data TypeTriggerDirectionMethod
InventoryWebhook (real-time)Uni- or bidirectionalDelta computation with fetch-on-notify
Product ContentWebhook + scheduledUnidirectionalTitle, description, SEO, options, variants
PricesWebhook or ERP importUnidirectionalBase price + compare-at price per variant
ImagesContent syncUnidirectionalURL + alt text, per-variant images

Architecture

🔌

Webhook Gateway

Receives Shopify webhooks with HMAC verification (timingSafeEqual). Returns 200 immediately, deduplicates by webhook ID (48h TTL), then queues async jobs per sync pair.

Express.jsHMAC Auth4 Webhook Topics

Job Queue & Workers

BullMQ Redis-backed queue with concurrency 5. Exponential backoff retry (4 attempts). Cost-based Shopify rate limiting (1000-point bucket, 50pts/sec restore).

BullMQRedisRate Limiter
🗃

Database & ORM

12 Prisma models: Store, SyncPair, ProductMapping, VariantMapping, InventoryLevelCache, LocationMapping, SyncAction, SyncLock, and more. Full audit trail.

Prisma 6PostgreSQL12 Models
📊

Observability

Prometheus metrics (webhook counts, sync duration, API latency, queue backlog). Winston structured logging with SSE live streaming. Slack alerts for failures and memory.

PrometheusWinstonSSESlack

Key Features

Echo-Loop Prevention

Dual-layer sync locks: in-memory Map (5-min TTL) + DB SyncLock (30s/120s TTL). Prevents infinite webhook loops between bidirectional stores.

Reconciliation Engine

Full inventory audit across all mappings. Source-wins authority. Detects quantity discrepancies, untracked items, and status mismatches. Dry-run + batch apply.

ERP Integration

Excel/CSV file import with configurable column mapping. Barcode-to-SKU lookup, warehouse quantities, retail/cost prices. Full and partial import modes.

Multi-Store Orchestration

Unlimited stores via OAuth installation. Many-to-many SyncPair relationships. Per-pair granular config: direction, filters, schedules, toggles.

Fetch-on-Notify Pattern

Never trusts webhook payload (may be null). Always fetches fresh state via Shopify GraphQL API, then computes delta against cached last-known quantity.

Monitoring Dashboard

24 RESTful API endpoints for dashboard control. Live log streaming via SSE, sync action history, product/location mapping editors, and settings management.

Smart Product Sync

Scheduled per sync pair via cron. Handles create, update, delete operations. SKU-based variant mapping. Smart sync: only update if source is newer.

Cost-Based Rate Limiting

Shopify-native 1000-point bucket with 50pts/sec restore rate. Per-shop rate limiter prevents API throttling across concurrent sync operations.

Automated Alerts

Slack notifications for queue backlog, failed syncs, and memory usage thresholds. Daily email digest via Resend with operation summaries.

Technology Stack

TypeScript Express.js Prisma 6 PostgreSQL BullMQ Redis Shopify GraphQL API Prometheus Winston node-cron Resend ExcelJS Vitest Docker

Deployment

Application

Dockerized Express.js server. Auto-deploy via GitHub Actions. Health endpoint at /api/status.

DockerPort 4001

Database

PostgreSQL with Prisma migrations run as release_command on deploy.

PostgreSQLPrisma Migrate

Queue Backend

Redis instance for BullMQ job queue, sync lock fast path, and rate limiter state.

RedisBullMQ