Skip to content
Hashlist logo

Expert network · Paid AI evaluation tasks for engineers

Hashlist

Hashlist, a Helsinki engineering marketplace (Gigarator Oy, founded 2020), runs a Data Operations Platform where engineers and domain experts take briefs from AI labs and build tasks for them: an agent prompt, a sandboxed environment, reference solutions, deliberately wrong answers, grading criteria and verifier tests. Its task templates are embedded and automotive software (ARM Cortex-M boards under QEMU and Renode, Eclipse automotive tooling) and desktop apps, and it also runs computer use briefs that pay by the hour to record yourself working in professional software (its app, read 2 Nov 2026). Signup is open, and each brief states its own pay before you start.

Per accepted task, or per hour of accepted screen recording; each brief sets its rate

Platform-stated

Join Hashlist (opens in new tab)How to get accepted

Is Hashlist legit?

Researched Nov 2026 · 7 sources

Mostly. Hashlist is a registered Finnish company (Gigarator Oy, since 2020) with carmaker clients, and its data platform's terms spell out how tasks are reviewed and paid. The platform is new, so its payment record is unproven. You are paid only when a reviewer accepts your task: rejected work is not paid and Hashlist keeps it, and each withdrawal is sent by hand, possibly after ID and tax checks. It pays Per accepted task, or per hour of accepted screen recording; each brief sets its rate (platform-stated).

Joined through its own application (it publishes no job postings)

Who is behind it

Gigarator Oy, trading as Hashlist, is registered in Helsinki (business ID 3153482-2, on the Finnish trade register since 14 September 2020). Its site says 100,000+ domain engineers are registered and names Stellantis, Toyota Motor Europe, Applied Intuition, and MobileDrive and Foxconn as selected clients. Its startup profile lists about EUR 1 million of seed funding and 1 to 10 staff.

Getting paid

Each brief states its US dollar pay: per accepted task, or per hour of accepted active time on the newer screen-recording briefs. Pay lands on your in-app balance; you withdraw to PayPal or a bank account (IBAN) and an administrator sends it by hand, aiming for 10 business days. A USD 50 minimum and ID or tax checks may apply.

Worker sentiment

Its Trustpilot page has 0 reviews (unclaimed, read 2 Nov 2026); no written worker review of the data platform found

Who can join

Worldwide for adults who can legally take paid independent work, except sanctioned countries; Hashlist may limit the platform to the countries where it offers it (DOP Terms section 3, last updated 25 Sep 2026, read 2 Nov 2026).

Fake Hashlist recruiters

We found no reports of scams using Hashlist's name (searched 2 November 2026). The paid-task platform is dop.hashlist.com and account questions go to team@hashlist.com. Nothing in its terms charges you to join or take a brief, so never pay anyone for access.

What people praise

  • Every task gets a written reason from Hashlist's reviewer, whether it is accepted or rejected, so you learn why.
  • Signup is free and open with an email or a Google account; a new account goes straight into the app, with no application form first (tried 14 Oct 2026).
  • The company behind it has run an engineering marketplace since 2020 for carmakers such as Stellantis and Toyota.

What people complain about

  • Rejected tasks are not paid, and the reviewer may credit less than the brief stated, for example for partial work.
  • Hashlist owns everything you build from the moment you create it, including the work in a task it rejects.
  • Withdrawals are reviewed and sent by hand, may wait for ID, tax or payment-account checks, and can carry a USD 50 minimum.
  • The data platform is new (its terms date from 25 September 2026); on 2 November 2026 its Trustpilot page had 0 reviews and we found no written worker review of it.
Who it suits

Best for: Embedded, automotive and systems engineers who can build a complete AI evaluation task (environment, reference solution, verifier tests) and want to be paid per accepted task. Not for: Anyone who wants pay for time spent whatever the outcome (only accepted work is paid), or a platform with a long record of paying contributors.

Honest catches

  • Pay here is platform-stated, not measured live from a job feed, so treat it as a benchmark rather than a guaranteed rate.
Sources for this review

What Hashlist pays

Platform-stated · as of 2026-11-02
Per accepted task, or per hour of accepted screen recording; each brief sets its rate

Researched rate card, platform-stated, as of 2026-11-02

Hashlist publishes no rates outside its signed-in app (its public pages at dop.hashlist.com, read 14 Oct, 29 Oct and again 2 Nov 2026). Its app shows two kinds of brief (read 2 Nov 2026 from the app's public code): a US dollar amount per successful submission (or a custom rate) with its own estimate of the hourly equivalent, and computer use briefs that pay an hourly rate for accepted active time recording yourself working in professional software, with an optional bonus per hour for narrating. Pay is earned only when a Hashlist administrator accepts the work (DOP Terms of Service, last updated 25 Sep 2026, sections 8.1 and 8.5, read 2 Nov 2026). A job aggregator, aitrainer.work, lists 9 open Hashlist briefs (read 2 Nov 2026): 6 embedded, hardware and controls briefs at $300 per accepted task with Hashlist's estimate of 3 to 4 hours, and 3 screen-recording briefs (Blender 3D design, 3D modelling and video editing) at $60 an hour of active time; those figures are a third party's copy of the briefs, not shown on Hashlist's own public pages, so the card states no number.

RoleResearched pay
AI evaluation task builder (embedded and automotive software)Per accepted task, set per brief
Reference solutions and verifier tests for AI tasksPer accepted task, set per brief
Screen recording of professional software work (computer use briefs)Per hour of accepted active time, set per brief

See how researched rate cards are compiled and graded.

Live Hashlist roles

Hashlist does not publish a public job feed.

You apply on Hashlist directly, then get matched to work from your account.

Join Hashlist (opens in new tab)

Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you.

Get new Agentic & RL Envs roles by email

Hashlist has no public job feed we can watch, so we'll email you new Agentic & RL Envs roles from the platforms that do, with real pay. Free, unsubscribe anytime.

The work Hashlist hires for

Each links to what the work involves, who hires for it, and what it pays across the market.

Agentic Tasks & RL EnvironmentsCoding & Software Evaluation

New to this? Start with the learning guides or compare pay across every platform.

How to get started on Hashlist

Researched Nov 2026

Create a free account at dop.hashlist.com with an email and password or with Google, take a brief from the work on offer, build its task in the app (the prompt, the files, your correct answer, the near misses and the scored steps), press Proof and then Publish; Hashlist runs several AI models against it, a reviewer accepts it or sends it back with a written reason, and an accepted task's pay lands on your balance, which you withdraw to PayPal or a bank account.

Who can join

  • Be at least 18, or the age of majority where you live if that is higher, and able to enter a binding contract.
  • Be legally able to do paid independent work and receive payment in your country, and not be in, or a national or resident of, a country under comprehensive sanctions. Hashlist may limit the platform to the countries where it makes it available, and it publishes no list.
  • One person, one account. Accounts are personal and may not be shared, and creating several, including to take the same brief twice or to refer yourself, is prohibited.
  • Hands-on expertise of your own. A task is one job from your own work, and Hashlist rejects puzzles and textbook exercises.

Step by step

  1. 1

    Create your account

    Sign up with an email address and password or with your Google account. There is no application form or interview first: a new account goes straight into the app.

  2. 2

    Take a brief

    Your dashboard lists the work on offer, put up by Hashlist and paid on acceptance. Each brief states the work required, the payment offered (an amount per successful submission, or a custom rate), whether it takes several submissions or only one, and how the task will be evaluated, such as which models it is run against and how many trials. Read the full brief, then press Take this brief: the task appears on your dashboard.

  3. 3

    Find a real job from your own work

    A task is one job from your own work, written up so an AI model can attempt it, and your checks can score the attempt. Hashlist recommends starting from a time a model gave you a wrong answer, hallucinated or failed to finish: a calculation from a datasheet it misread, a configuration file that would fail on the real tool, or a review comment you had to write on code it generated.

  4. 4

    Write the prompt, the golden set and the scored steps

    The app creates the task folder for you. Write the prompt the way the job would reach you, then the Golden set: your correct answer, one or more near misses (the wrong answers a capable colleague would give), and your steps. Each scored step is an outcome worth 1 to 5 points, checked by a value between two numbers, a line that contains a phrase, or a judge model that reads the answer.

  5. 5

    Run Proof, then Publish

    Proof runs your checks against your right answer, your wrong answer and an empty output, and shows which check caused a failure; a near miss you ticked under a step also has to fail that step. When Proof passes, press Publish. A task you hand in is locked while it is in review.

  6. 6

    Model runs, then a reviewer

    Hashlist runs several models against your task, with your prompt and with a rewritten one, with and without tools; the task passes if some attempts succeed and some fail. A reviewer then reads it, checks the files for confidential material and tries to break your checks. If something needs changing, you get a note on the task in the app, and most fixes take a few minutes.

  7. 7

    Get paid, then take the next brief

    Every task gets a written reason from the reviewer. An accepted task's pay lands on your balance at the amount the brief offered when you handed it in (the reviewer may credit a different amount, for example for partial work, and records why). A rejected task is not paid.

No entrance test: every task you hand in is run against AI models and reviewed

There is no entrance exam or interview. Your first task is the test: every task you hand in is run against several AI models and then reviewed by a Hashlist administrator, who accepts or rejects it and gives a written reason. A good task often scores somewhere in the middle: if every model attempt fails it is too hard, and if every attempt passes it is too easy. It must also be real work, not AI-generated, hard to cheat, give the same score every run, and come with a realistic wrong answer.

Format

A task you build in Hashlist's app: the prompt, the files the job needs, your correct answer, near misses and scored steps worth 1 to 5 points each, checked by Proof before you publish, then by model runs and a human reviewer. Its full guide also describes building a task by hand, with an environment, a verifier and reference answers.

How to prepare

  • Start from your own working week: think of a time you asked a model to do something and it gave back the wrong answer, hallucinated or failed to complete it.
  • Name the realistic wrong answer, the mistake a capable colleague would make, such as the wrong unit or a clause from an old spec revision.
  • Make it hard to cheat and repeatable: the model must be unable to edit your checks or find the answer lying in the files, and your checks must give the same result every run (pin versions, avoid timing and network calls).
  • Write a step as an outcome, not an action: 'uses the worst-case sleep current, not the typical one', not 'opens the datasheet'. When a step is judged, your correct answer must write that reasoning into the answer file in plain sentences.
  • Read the full guide at dop.hashlist.com/guide before your first task: it covers every field and the rules the app will not repeat, such as stating tolerances, not equality, and the band you would accept from a colleague.

Why people get rejected

  • An AI-generated task: Hashlist says all AI-generated submissions are automatically declined (it recommends using AI on less than 20% of a task; the builder's own Ask helper and check suggestions are fine).
  • A puzzle or textbook exercise instead of a job from real work.
  • A task the models always pass, or always fail.
  • Checks a model can get around, or an answer left lying in the files.
  • Material you have no right to use, such as an employer's confidential code or data, or a task copied from a public benchmark.
How long it takes

Hashlist publishes no review time. Signing up is immediate and the pace of building a task is yours; after you publish, Hashlist runs the models and a reviewer decides. A task sent back comes with a note in the app, and most fixes take a few minutes.

When you get paid

Each brief states its payment in US dollars, fixed at the amount it offered when you handed the task in, and you are paid only when an administrator accepts the task: a rejected task is not paid. Accepted pay lands on your balance. You request a withdrawal to a PayPal address or a bank account (IBAN), and an administrator reviews and sends each one by hand: the Terms say Hashlist aims to process requests within 10 business days, may require identity, tax or payment-account verification first, and may apply a minimum withdrawal of USD 50. Your provider's fees and currency conversion are yours.

Tips to get accepted

  • Use numbers that come from somewhere: Hashlist's guide asks for the spec section, table or measurement you would cite to defend the limit, and an hour of your working day with the real data.
  • Make the near miss wrong, not broken. A file that produces no answer scores zero for the wrong reason; the near miss should be the mistake a good engineer makes under time pressure.
  • Keep briefs out of outside AI tools. The Terms treat unpublished briefs, your tasks and reviewer feedback as confidential, including entering them into third-party AI tools.
  • Have your tax form ready before you withdraw: Hashlist may ask for IRS Form W-9 or W-8BEN, or a local equivalent, before paying you.
  • Hashlist owns every task from the moment you create it, including one it rejects, so build from your own experience and never from an employer's confidential material.
  • Hashlist also posts computer use briefs: with its DOP Recorder desktop app (macOS 12 or later, Windows 10 or later) you record yourself doing a real job in professional software while explaining it, cut anything private, and upload it. Accepted active time is paid by the hour, a brief may add a bonus per hour when you talk it through, and it can reject a recording with too few minutes of active work.

What isn't public

  • Hashlist publishes no rates outside its signed-in app: each brief states its own payment there, and none of Hashlist's own public pages lists the open briefs (read 2 Nov 2026).
  • It publishes no acceptance rate and no review time.
  • Its own pages differ on payouts (all three read again 2 Nov 2026): its Terms (last updated 25 September 2026) say an administrator sends each withdrawal by hand to PayPal or a bank account, aiming for 10 business days; its signed-in Balance page says 1 to 3 business days; and its first-task guide says balances are paid out weekly through Stripe.
  • It names no list of the countries it pays, only that it may limit the platform to the countries where it makes it available.
  • Its Terms (section 6.3, last updated 25 September 2026) say a rejected task may be revised and handed in again, while its app (read 2 Nov 2026) also marks some tasks Rejected with the note that they cannot be handed in again.
  • The data platform is new (its Terms date from 25 September 2026); on 2 November 2026 its Trustpilot page had 0 reviews and we found no written worker review of it.
Start your Hashlist application (opens in new tab)
Sources (6)

From the forum

Ask the people doing Hashlist work what only they can tell you.

Ask about Hashlist

Frequently asked questions

Is Hashlist legit?

Mostly. Hashlist is a registered Finnish company (Gigarator Oy, since 2020) with carmaker clients, and its data platform's terms spell out how tasks are reviewed and paid. The platform is new, so its payment record is unproven. You are paid only when a reviewer accepts your task: rejected work is not paid and Hashlist keeps it, and each withdrawal is sent by hand, possibly after ID and tax checks.

Is Hashlist a scam?

No. Hashlist is a real company. We found no reports of scams using Hashlist's name (searched 2 November 2026). The paid-task platform is dop.hashlist.com and account questions go to team@hashlist.com. Nothing in its terms charges you to join or take a brief, so never pay anyone for access.

Does Hashlist pay?

Yes. Hashlist is an established engineering marketplace (Gigarator Oy, Helsinki) whose supplier side serves carmakers and mobility firms such as Stellantis, Toyota Motor Europe and Applied Intuition (its site, read 29 Oct 2026). Its data platform for AI tasks is new and has no written worker reviews yet (Trustpilot: 0 reviews, read 2 Nov 2026), so its payment record is unproven. Payouts: Withdrawal on request to PayPal or a bank account (IBAN), reviewed and sent by hand within about 10 business days; a USD 50 minimum may apply, and fees and currency conversion are yours (DOP Terms section 9, last updated 25 Sep 2026, read 2 Nov 2026). Its own app gives other times: its Balance page says 1 to 3 business days and its first-task guide says weekly through Stripe (read 2 Nov 2026). Each brief states its payment in US dollars, fixed at the amount it offered when you handed the task in, and you are paid only when an administrator accepts the task: a rejected task is not paid. Accepted pay lands on your balance. You request a withdrawal to a PayPal address or a bank account (IBAN), and an administrator reviews and sends each one by hand: the Terms say Hashlist aims to process requests within 10 business days, may require identity, tax or payment-account verification first, and may apply a minimum withdrawal of USD 50. Your provider's fees and currency conversion are yours.

How much does Hashlist pay?

Per accepted task, or per hour of accepted screen recording; each brief sets its rate. Hashlist publishes no rates outside its signed-in app (its public pages at dop.hashlist.com, read 14 Oct, 29 Oct and again 2 Nov 2026). Its app shows two kinds of brief (read 2 Nov 2026 from the app's public code): a US dollar amount per successful submission (or a custom rate) with its own estimate of the hourly equivalent, and computer use briefs that pay an hourly rate for accepted active time recording yourself working in professional software, with an optional bonus per hour for narrating. Pay is earned only when a Hashlist administrator accepts the work (DOP Terms of Service, last updated 25 Sep 2026, sections 8.1 and 8.5, read 2 Nov 2026). A job aggregator, aitrainer.work, lists 9 open Hashlist briefs (read 2 Nov 2026): 6 embedded, hardware and controls briefs at $300 per accepted task with Hashlist's estimate of 3 to 4 hours, and 3 screen-recording briefs (Blender 3D design, 3D modelling and video editing) at $60 an hour of active time; those figures are a third party's copy of the briefs, not shown on Hashlist's own public pages, so the card states no number.

Is Hashlist worth it?

Hashlist pays Per accepted task, or per hour of accepted screen recording; each brief sets its rate (platform-stated). It rewards credentialed experts who clear its assessment far more than open generalist work. Worker sentiment: Its Trustpilot page has 0 reviews (unclaimed, read 2 Nov 2026); no written worker review of the data platform found.

How does Hashlist pay out?

Withdrawal on request to PayPal or a bank account (IBAN), reviewed and sent by hand within about 10 business days; a USD 50 minimum may apply, and fees and currency conversion are yours (DOP Terms section 9, last updated 25 Sep 2026, read 2 Nov 2026). Its own app gives other times: its Balance page says 1 to 3 business days and its first-task guide says weekly through Stripe (read 2 Nov 2026).

Where can you work for Hashlist?

Worldwide for adults who can legally take paid independent work, except sanctioned countries; Hashlist may limit the platform to the countries where it offers it (DOP Terms section 3, last updated 25 Sep 2026, read 2 Nov 2026).

How do I get started on Hashlist?

Create a free account at dop.hashlist.com with an email or Google, take a brief from the work on offer, and build its task in the app: the prompt, the files, your correct answer and near misses, and the scored steps. Press Proof, then Publish. There is no separate application or interview.

Does Hashlist have a test or an interview?

No. Each task you hand in is run against several AI models, then reviewed by a Hashlist administrator, who accepts or rejects it and gives a written reason.

Can I hand in a Hashlist task again after it is turned down?

It depends on how it was turned down. Hashlist's app marks a task that fails review either Fail, sent back with notes, which you can fix and hand in again, or Rejected, which cannot be handed in again. Neither is paid.

Can I use ChatGPT or Claude to build a Hashlist task?

No. Hashlist says AI-generated tasks are rejected: the task, its environment and its gold answers must come from your own hands-on engineering. The builder's own Ask helper and check suggestions are allowed.

How do I get paid by Hashlist?

Accepted pay lands on your balance. You request a withdrawal to PayPal or a bank account (IBAN), and an administrator sends it by hand; the Terms aim for 10 business days and allow a USD 50 minimum and identity or tax checks first.

Who can join Hashlist?

Anyone 18 or over who can legally do paid independent work and receive payment in their country, outside countries under comprehensive sanctions, with one account per person.

Compare Hashlist with

AfterQuery logoAfterQuery

Expert reasoning and rubrics for frontier labs, weekly Stripe pay.

$15–$175/hr; middle half $50–$104/hrPlatform-stated
AITraining.jobs projects logoAITraining.jobs projects

Paid expert projects we run ourselves for companies building AI: professionals grade, write and compare AI output in their own field, paid for every hour, onboarding included.

Set per project, written before you acceptPlatform-stated
Aligned Labs logoAligned Labs

Expert network for frontier AI labs (OpenAI's logo on its home page) where experts set their own hourly rate for RLHF and evals.

$40-$150/hr (self-set; top experts $150+)Platform-stated
Alignerr logoAlignerr

Design the training tasks in your own professional domain.

$10–$200/hr (German voice actors to $280/hr)Platform-stated

Ordered by what applying is worth, lifted by a referral boost for the platforms that pay us a referral fee; it never changes what a platform pays you.

See the full pay comparison

Short of experts at Hashlist? We find the experts (you contract them yourself) or we run the project (we deliver the expert work).