We asked 15 hiring design managers to rate 300 applications.
Applicants responded to the unified description of a product designer job.
No limitations on how and what the applicants were writing about themselves. They could publish any contact information they wanted, any links and an unlimited amount of text.
- You have approximately 50 seconds of attention, including your cover letter (or CV) and portfolio. Ask your friend to read it out loud. Time it. Don't be embarrassed, reiterate with different people.
- You would probably want your portfolio to be accessible from mobile devices, because of how information about your application spreads among hiring people. There are scenarios when the recruiter forwards your info via email to a hiring manager, and the latter opens it on a mobile phone (like we all do).
- You would really want to show your accomplishments (finished tasks) and your understanding of processes. It is not about keywords; it is about how you use them because humans are still able to spot some bullshit pretty easily. Describe what was done instead of a process.
Experts research (hiring managers)
Hiring managers were from the tech companies like Acronis, Arrival, Intercom, Miro, Sberbank, Yandex, Revolut, Raiffeisenbank and like.
They were seven women and eight men; they were making decisions on hiring people for their departments, and they've all shared one field of expertise: digital product design.
Experts could answer "Yes," "No," or mark an application as "Spam."
Here's how the UI looked like:
The algorithm and research process
Each application was rated by at least seven experts. The algorithm distributed applications evenly between women and men (experts) four times, and three times between random experts, to even the odds.
Each applicant received a weighted score of their chances of being invited to an interview, based on the number of upvotes to downwotes proportion and lookalike analysis (a dash of machine learning).
After we've got initial applications rated by experts, we invited applicants to rate each other and compared the results. On average, applicants downvote each other 25% more, so non-experts ratings are lower.
The key finding was that applicants learned a lot about how to write better introductions and what was important and what was not. They've learned that by doing, not theory.
The other important thing was that a lot of people suffer from zero feedback after they apply. Lack of feedback of any kind prevents good candidates from growing professionally and builds frustration.
This exercise gave them at least some feedback in the form of stats, and they could learn from the community (other applicants) what is important in their own stories and what is not.
- On average, it takes 50 seconds to read, review, and make a yes-or-no decision.
- Only 3% of all applicants would've been accepted by all hiring managers (100% approval rating);
- 8% of applicants got an 80% approval rating
- 25% of applicants got an approval rating between 50% and 80%.
- The rest 64% of all applicants received negative approval ratings.
Test your cover letter for free
We've updated our research platform into the public web service and you can apply your story and read/rate others, get your stats in return.