RepWise

A fitness web app that scores squat and push-up form from a webcam, speaks corrections aloud, tracks macros, and generates workout plans.

HackByte 3.0

🎓 IIITDM Jabalpur

📌 Jabalpur, Madhya Pradesh

🚀 36 hours Hackathon Project

🏅 Track

What it is

RepWise is a fitness web app built for a hackathon by a team of four. It analyzes exercise form from a webcam or an uploaded video, generates a workout plan from an onboarding profile, and logs nutrition by scanning barcodes or photographing nutrition labels. Users can also compete on rep scores in live video rooms.

The problem

HackByte 3.0 asked teams to build free alternatives to premium services behind paywalls and red tape. The team chose fitness. Form coaching, personalized programming, and nutrition logging usually sit behind subscriptions. RepWise combines all three in one app.

How it works

RepWise is a SvelteKit app. Pages and API routes live in one codebase, and Prisma talks to PostgreSQL. The table lists the main modules.

ModuleWhat it doesWhere
Form correctionRuns MediaPipe pose detection in the browser, scores each rep, and speaks feedbacksrc/lib/detection-component.svelte, src/lib/exercise-feedback.ts
Voice feedbackConverts a feedback sentence to speech with ElevenLabssrc/routes/api/voice
Workout plansAsks Gemini for a structured plan that matches the onboarding profilesrc/routes/api/workout-plan, src/lib/toolspecs/fitness-planner.ts
Macro trackingReads barcodes in the browser, looks products up in Open Food Facts, and stores macrossrc/routes/(app)/macro-tracking, src/routes/api/macros
Label OCR (optical character recognition)Sends a photo of a nutrition label to Mindee and stores the parsed resultsrc/routes/api/ocr
CompetitionsStreams video through Agora and stores scores per user and roomsrc/routes/(app)/compete, src/routes/api/competition
Heart rateAccepts heart-rate samples from an authenticated clientsrc/routes/api/heartrate
AuthEmail and password plus Google sign-in through Better Authsrc/lib/auth.ts

The data flow for form correction stays on the device until feedback needs speech:

  1. The page loads the MediaPipe pose_landmarker_heavy model from static/ and runs it on the video with the GPU delegate.
  2. Each frame appends a landmark result to a sequence.
  3. Every 10 frames, the exercise's feedback function reads the sequence and returns messages, a score, and a rep count.
  4. The component speaks the latest message through /api/voice, at most once every 5 seconds.

What I built

I built the pose-based squat and push-up feedback with rep counting, the spoken corrections, and the competition room, scoring and status APIs and interface. Jay built authentication and onboarding, the dashboard, the workout planner and chat interface, barcode enhancements and the move to Agora. Sundaram built the Prisma schema, macro tracking, nutrition-label OCR, the routine and heart-rate APIs, and the theme and sidebar.

Design decisions

Rule-based form scoring. The code computes joint angles from landmarks and compares them with fixed thresholds. A squat scores 25 points each for depth, knee alignment, hip-to-knee balance, and tempo. A push-up uses depth, body alignment, arm position, and tempo. This approach needs no training data and produces a specific message for each failed check. The cost is coverage: only two exercises exist, and the thresholds apply to every body type.

Pose inference in the browser. Landmark detection runs client-side, so the server never receives video for form analysis. The trade-off is that performance depends on the user's device and GPU.

Structured output from Gemini. The workout route declares a create_workout_plan function with a typed schema (days, exercises, sets, reps, load, rest). It reads the arguments of the model's function call instead of parsing free text. The plan then has the same fields (sets, reps, load, rest) that the Routines table stores.

Managed video for competitions. The commit history shows that on 2025-04-05 the team added its own room and signaling API, then removed it and integrated the Agora RTC SDK the same day. Agora carries the video. The app stores only room status and per-user scores in PostgreSQL.

Run it locally

You need Node.js 20, a PostgreSQL database, and API keys for the services above.

  1. Clone the repository and install dependencies. The install step also runs prisma generate.

    git clone https://github.com/Crew-object-Object/RepWise.git
    cd RepWise
    npm install
    
  2. Create a .env file with these variables: DATABASE_URL, PUBLIC_BASE_URL (for example http://localhost:5173), GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET, GEMINI_API_KEY, ELEVENLABS_API_KEY, OCR_API_KEY, and PUBLIC_AGORA_APP_ID.

  3. Create the tables.

    npx prisma db push
    
  4. Start the dev server.

    npm run dev
    

Reference

AreaChoice
FrameworkSvelteKit 2, Svelte 5, TypeScript
UITailwind CSS, shadcn-svelte components, bits-ui
DatabasePostgreSQL through Prisma
AuthBetter Auth
Pose detection@mediapipe/tasks-vision
Text to speechElevenLabs
Workout plansGoogle Gemini (gemini-2.0-flash)
Barcodes and nutrition databarqode, Open Food Facts
Label OCRMindee
Live videoAgora RTC
Deployment@sveltejs/adapter-vercel
TestsPlaywright end-to-end test, run in GitHub Actions against a Postgres service

The Prisma schema defines User, Onboarding, Routines, Macros, HeartRate, CompetitionRoom, and CompetitionScore, plus the Better Auth tables.

Outcome

The team built RepWise at HackByte 3.0 at IIITDM Jabalpur on 2025-04-04. The hackathon page records a track placement and the GitHub award for issue and pull request tracking and documentation. The repository has 163 commits across three contributors. The first commit is dated 2025-04-04 and the last 2025-04-06. A deployment is listed at rep-wise.vercel.app.

Liked RepWise? Here's my background.

Full-stack TypeScript, shipped to real users.