Early-stage dataset marketplace startup
On-device scanning and encryption for a dataset marketplace that never holds a file
An early stage startup wanted a trusted marketplace for AI training data that never holds a file. A desktop agent we built scans and encrypts each dataset on the seller's machine, the file stays in the seller's cloud drive, and the platform keeps only manifests, hashes and licenses.
Data & AnalyticsNon-custodial dataset marketplaceBuilt from zero
The challenge
Buyers of data for AI could not check origin or quality before paying, and existing exchanges took custody of the data, and the legal liability with it.
What we did
A web marketplace, an admin console and a desktop agent for Mac, Windows and Linux that scans each dataset for malware, unsafe content and personal data, encrypts it locally and lists only its manifest.
The outcome
The MVP was built end to end and demonstrated to the client, with a production environment, a refund flow and desktop installers for three operating systems.
- What shipped
- Web marketplace, admin console and desktop agent
- Checks on the device
- Viruses, malware, unsafe content and personal data
- Platform holds
- Manifests, metadata, hashes and licenses only
- AI flags
- Reviewed by a human admin, who can override
The starting point
A new two sided marketplace where individuals and organizations buy and sell datasets for AI, machine learning and research, from shopping history and app usage to IoT sensor data. The client, an early stage startup, brought a concept, a license framework, quality attributes with pricing multipliers and an early pricing calculator.
What had to change
Existing exchanges take custody of the data, give little pricing transparency and do not let buyers check origin and quality before they pay. The marketplace had to earn trust from both sides without ever holding the data itself.
What could not break
- The platform stores only manifests, metadata, hashes and license agreements, never dataset files
- No upload, purchase or download before identity verification is approved
- Files up to 10 GB each in the MVP, with a path to 1 TB
- Every sale needs a legally binding license, generated automatically
- Large transfers must pause, resume, retry and verify their integrity
The calls we made
We settled data custody on the first discovery call: the platform would never hold a dataset. Every later decision, from where checks run to how files move, follows from that line.
- 01
Validate on the seller's machine
The desktop agent checks each dataset on the device, with virus and malware scanning, rule based detection of unsafe content, a local personal data scanner for items such as social security and card numbers, duplicate and format checks, and metadata extraction. Raw files never reach the platform, which keeps the legal and compliance risk down.
Ruled outRunning validation in the cloud on uploaded copies of the data.
- 02
Keys and files stay with the seller
The agent generates an AES-256 key on the seller's device, encrypts there and places the file in the seller's own Google Drive, Dropbox or OneDrive. The buyer decrypts locally and verifies integrity against the listed hashes.
Ruled outKeeping files in the platform's own storage, which would have made the marketplace a data processor under GDPR.
- 03
Humans over AI flags
AI validation runs on every upload, and anything it flags goes to an admin who reviews it and can override the result. Users carry statuses with timestamps, such as unverified or flagged, so trust and safety has a record to work from.
- 04
Hashes and signed licenses, not a blockchain
A unique hash per dataset blocks duplicates and proves integrity, and every purchase generates a license PDF that both parties sign, backed by their verified identities.
Ruled outA blockchain for immutability, rejected because it does not suit high volume data transfer.
- 05
Text chat over video
Buyers and sellers talk in text chat with image previews, which leaves an audit trail and costs less to run.
Ruled outVideo calls between buyers and sellers, first discussed for high value deals.
How the system fits together
What we built
Web marketplace
Identity checks, dataset listings with metadata and quality attributes, a pricing calculator, bounties and proposals, ratings and reviews, bookmarks, chat, and an exclusive sale flow that removes or watermarks the listing.
Desktop validation agent
An Electron agent for Mac, Windows and Linux that connects to Google Drive, Dropbox and OneDrive, accepts raw files only, runs virus, malware, unsafe content and personal data scans on the device, shows the seller what it found, then encrypts and uploads in resumable chunks.
Licenses and payments
Stripe Identity for identity checks and Stripe Connect for payments, payouts, refunds and webhooks, with a license PDF for every sale covering the dataset, the parties and the pricing and transfer terms.
Admin console
Identity review, AI flag review with override, user statuses with timestamps for trust and safety, refunds, invoices and seller and admin dashboards.
Transfer built for large files
Files up to 10 GB move in resumable chunks with pause, resume and hash based integrity checks, and the buyer decrypts and verifies each dataset on their own machine.
How it works
Verify
Buyers and sellers pass identity checks with a government ID and a live image before they can list, buy or download.
Scan and encrypt locally
The desktop agent scans the dataset on the device, shows the seller what it found, and encrypts the file with a key generated on the seller's machine.
List a manifest
Only the manifest, metadata and hashes go to the marketplace, while the encrypted file stays in the seller's own cloud drive.
Review flags
Anything the AI flags goes to an admin, who makes the call and can override it.
Buy with a license
Checkout generates a license both parties sign, and the buyer downloads, decrypts locally and verifies integrity.
A closer look
What changed
The marketplace, admin console and desktop agent were built end to end and demonstrated to the client, with a production environment, Stripe webhooks for staging and production, and desktop installers for three operating systems. No dataset file has to pass through the platform.
- Marketplace, admin console and desktop agent
- Shipped
- macOS, Windows and Linux
- Installers
- None, only manifests, hashes and licenses
- Files on the platform
Under the hood
- Datics role
- Product built by Datics
- Integrations
- Stripe Identity and Stripe Connect, Google Drive, Dropbox and OneDrive through OAuth, Twilio for phone checks and 2FA, SendGrid and later SES for email, and AWS S3 for licenses, receipts, previews and installers.
- Data boundary
- Raw datasets stay on the seller's device and cloud drive, encrypted with AES-256 keys made locally, and the platform stores manifests, metadata, hashes and licenses.
- Validation
- On the device: ClamAV virus scanning, YARA rules for malware and unsafe content, a local personal data scanner and a content validator, plus tampering and duplicate checks, a unique hash per dataset and raw files only. Formats supported: CSV, XLSX, JSON, MP4, PNG, JPG and XML.
- Deployment
- Separate staging and production environments, with production on AWS and desktop installers for macOS, code signed, and for Windows and Linux distributed from S3.
Cloud and infrastructure
Apps and APIs
Also in the stack
How it shipped
Discovery and prototype
Flow diagrams for buyers, sellers, admins and the agent, a clickable prototype of the complete flow, a security document and a scope document with 85 screens came before any build sprint.
Core build sprints
Sign up with Google login, OTP and 2FA, identity checks with admin review, the pricing calculator, cloud drive connections, listings, reviews, bookmarks, public key exchange and the Stripe purchase flow, with the Electron agent.
Agent QA and client demos
QA cycles on the Mac, Windows and Linux installers, chunked uploads, scan results, key decryption and macOS code signing, then bounties, invoices and chat, and live demos to the client.
Production readiness
A production environment, Stripe webhooks for staging and production, the refund flow, cookie based auth, the 10 GB limit and installers on S3.
Sprints with demos and reports
The build ran in sprints, with written sprint and milestone reports, live demos to the client, demo scripts and a playbook guide.
Decisions pinned in writing
Calls such as text chat over video, an exclusive price calculated on the back end, ratings in the MVP and enterprise analytics deferred went into the scope document early, and an AI bill of materials was shared with the client.
What this unlocked
The same local first design carried into an enterprise version, with a Rust and Tauri agent, watermarking, attestation and security gates from the first milestone.
Related work
Have a product with the same kind of constraints?
Tell us what already exists and what you want it to do. We will tell you what it takes.