This is very python specific. While I got a 15/15 and can easily answer these, the reason for that lies with the fact that I am very fluent in Python, had I not been I would have not gotten at least 6-8 of these wrong.

This isn't a generalist quiz, nor is it a good baseline for interviewing candidates for the hiring process, this wouldn't tell me anything about the application of their skillset.

Not entirely applicable to this particular quiz (most of the questions were relatively simple until the final few), but the interview questions like the ones that Hired and Triplebyte focus on honestly make me feel like an imposter.

I've been a software developer for over 4 years but I don't have a formal CS education. I've gone through CLRS along with MIT's open-courseware--but I rarely use any of that directly in the workplace and so have forgotten a lot of it. And I think I'm a pretty good software developer, yet in the past when I've tried to get into Hired or Triplebyte, I always leave feeling like I don't belong in this profession because I can't tell you the big-O complexity of DFS off the top of my head, much less write the algorithm on a scratch pad.

I've been doing technical interviews for a few weeks now (I was recently laid off) and I'm honestly just a little tired of the brain teasers and non-applicable algo questions--and more often than not, the interviewers flexing their "knowledge" muscles.

Case in point: I'm in final stages with multiple companies for a Rails position, yet only one of those had even mentioned Rails in the technical interviews--the others decided to quiz my CS knowledge and one of them even quizzed my FP knowledge (for a Ruby position…). I don't understand it.

This was an interesting quiz, but perhaps not the way the authors intended. The second half was straightforward first-year computer science, but the first half challenged me to speculate about the syntax of a novel scripting language.

The wording on question 9 (the regex one) is a little unclear: both b[l].e and ([^b]b[^b]*){3} will match blabber but not babel. Presumably the intended meaning is that the entire string must be matched, but the question does not make this explicit.

Cute quiz. Question 9 is incorrect, however – or at least poorly worded. It asks about regex matches and only considers the last one to match, while the second one also matches, albeit incompletely. The last regex is the only one that matches the entire input string, but that's not usually how regexes work – e.g. in Perl or Ruby. This verbiage seems to be specific to Python, which uses `re.match(str)` to mean "match the entire string" as compared to `re.search(str)` which means "search for any matching substring". Arguably a decent distinction, but very Python-specific.

[Update: this is now fixed in the quiz.]

Got a few easy ones that were about language implementation details wrong because I've never done Python. Only realised it was Python and not some kind of beautiful pseudocode when I kept getting answers wrong

We've been using a quiz as part of our hiring process (at Triplebyte) for the last two years. However, a lot of people do our quiz for fun. So we decided to create a version optimized for this (and hopefully a little education). The problems come from the most common mistakes / misunderstandings we see when giving interviews, and ramp up in difficulty. We put a lot of effort into writing interesting answers.

Let me know what you think!

"True programming generalists are rare. This quiz covers basic programming, web development, algorithms and low-level systems. The questions start easy, but get harder fast. Only 3% of programmers can get all the way through. Can you?"

This test doesn't cover any of that, makes me worry if they think it does.

it's more :-

do you understand python syntax?

do you understand a few algorithms?

do you understand a few security issues?

not sure how you'd go from this test to working out if a person is a programming generalist. In fact I'd be worried that someone passing this test would be fresh graduated from college where they used python.

imo, this is the right way to do interviewing. I'm currently interviewing with several companies, and one thing that stresses me out is the expectation that

1. candidate must discover algorithm to solve unseen problem

2. candidate must code up algorithm discovered in #1

3. candidate's code must work

4. steps 1,2,3 must happen under 1 hour

I think this is a bit too much, and completely misrepresents the current working state of professional programmers.

If I am asked to do #1, I usually do a literature survey - I read papers before rolling my own algo from scratch. 99% of the time, there's a good enough algo already out there.

If #2 - ok.

If #3 - not ok. Typically you need a debugger & 1-2 passes before you get it right.

If #4 - definitely not ok. Step 1 itself can take you forever, and you don't even know if its correct. Then you've got #2,#3...adding time constraint to this mix is crazy.

So yeah, this quiz has two highlights: showing a block of code & asking to fill in the blanks - much better, highly recommended. Also, looking at a block of code & figuring out what it does - definitely part of programmer's everyday work. I hope more companies adapt this technique.

I'm sorry but I just can't take a quiz/website/company seriously that assesses programmer ability, and yet they put the freaking answer key in Local Storage. Seriously, quizState? I got a 100%. ;)

{"questions":[{"name":"max_function","answerIndex":2,"question":"Fill in the missing line of code:","code": ...

This quiz is not a programming quiz but a Python quiz.

If you only fail the regex, unicode, and hash security question you ought to get a badge of innocence you can print and cut out

15/15. Some questions felt like they were trying to trap me. Or maybe I'm just tired, I've had a long day of work.

Congratulations on making this! It's less fun for me than algorithm puzzles, but I can see it working well for hiring purposes. I'd love to see some software company use it and report the results.

The quiz wouldn't pass security review, FWIW. The answers are shipped with the javascript, and validation is done client side, so they're just an 'inspect element' away.

On the last two questions, I'm curious why the ability to memorise which algorithm to use is a useful skill for a professional software engineer, let alone a useful interview question. Isn't the best answer "I would look up the best algorithm by ..."?

Concerning 12: timing attacks are not a relevant threat for hash function computations. See https://github.com/codahale/bcrypt-ruby/pull/43#issuecomment.... Supported by our own tptacek here https://twitter.com/tqbf/status/866115075110825984

undefined

Question for ammon : What's the time frame for expanding to remote jobs?

Question for other HN readers : How much did you score on the quiz? I scored 10/15 and it shows that I'm better than 50% of engineers i.e I'm somewhere right in the middle. Not very encouraging, but if I brushed up on the graph algorithms, I could have done better.

Seems extremely language-semantics specific.

In the 3rd question, if you give the wrong answer, it says Java is pass-by-reference as well, though IIRC it is always pass-by-value. Also JS passes primitives by val as well if I'm not mistaken.

> claims this is a generalist programming quiz

> first half focuses entirely on weird Python quirks.

Wew.

On the plus side, I still somehow managed to get 9 out of 15 right, which apparently puts me above 50% of all programmers.

Mildly annoyed at myself for getting question 8 wrong: the true answer was what I thought that code would be doing, but the bytes didn't look like valid UTF-8. I had a brainfart and thought that continuation characters were always valid ASCII, so a string containing only characters >= 0x80 wouldn't be valid.

The (much saner) reality is that continuation characters are never valid ASCII.

10 I admit I got through elimination: three answers were obviously wrong, one could have gone either way.

I'm not very familiar with Python but I liked this quiz. I got two wrong (13/15), once for a stupid mistake and once for not knowing the algorithms well enough. In a couple cases I didn't know the right answer, but was able to determine that all other answers were wrong, which worked.

Overall, I think this is a good level of knowledge that isn't too nitpicky but has enough meat to really test if someone knows something.

This was fun. It's been a while since I took a multiple-choice test, and it called to mind the Cluss Test <https://www.recompile.se/~teddy/cluss-test>. This test was fairly cluss-resistant, as I don't recall making any inferences from the test's format or language rather than its content.

14/15 - I missed the one where Python doesn't copy the row before appending it to the matrix.

This is a quiz about python gotchas.

I first found out about Ammon and Triplebyte through this YC video interview. I thought it was interesting, so I'll share:

https://www.youtube.com/watch?v=haxZDcTWdOo

Too many language specific aspects.

Reminded me why I dislike Python...

Where do the percentiles come from that are displayed at the end of the quiz? Are these based on a previous test sample group or are they being updated as more people take the quiz?

Is there a JS equivalent of something like this out there?

What's the deal with the warp searching question? There are both depth and breadth first algorithms that can handle cycles. Bit annoying since the example I had in mind was Djikstra's but it turned out that was classified as a BFS instead of DFS like I thought it was. Finding the shortest path from A to D in 4 steps for example seemed more "depth" to me, but maybe the algorithm as a whole doesn't neatly fit into either.

Is this intended to be taken with access to references and a python interpreter, or closed book?

I only got 50 percentile which I thought wasn't bad for not knowing python

I would have done better had I known how python does parameter passing...

undefined

I took the Triplebyte quiz on Facebook and after finishing, they didn't even bother giving the score. It was very outputting, and felt like a trap, asking for an email address instead of giving the score. I won't ever be doing business with them.

I like the idea, but my first reaction was that this is a test of how well you know Python's quirks (like the problems with not deep-copying things).

But then I realised that that's not true. For example, call-by-value and call-by-reference are pretty pervasive across languages: many of those examples would've worked the same in Java, say. It's not familiarity with Python, but rather familiarity with a certain kind of language (which admittedly constitutes the majority of those used in industry ATM) that the test demands, at a time when even Java is getting nice FP-like abstractions and the JavaScript community is cooking up all sorts of ways to bring the benefits of that kind of thinking to their own work (from React/Redux all the way to Ramda and the like).

If you're more used to languages that "get out of the way" by eliding what could very well be termed "implementation details" (e.g. is there any reason[0] to use call-by-reference in the absence of global mutable state given a "sufficiently smart optimizing compiler"?), I'd predict (n = 1) that you'll be more likely to make mistakes.

I would've done far better on this quiz back when I was playing with matplotlib as an eighth- or ninth-grader: apart from the token map implementation, the quiz really favors knowledge of counterintuitive behavior common in "mainstream" imperative languages over an understanding of "composability" that users of modern functional languages are used to. (And no, that doesn't only mean Haskell or "ultra-Haskells" like Idris, but also languages like OCaml, F#, and so on.)

It would be much nicer if the quiz could be less about the candidate coaxing her preferred meaning out of a language implementation and more about testing the meanings a candidate knows how to implement with the help of the language.

[0]: For instance, the standard reaction to things like

    sum (map (\x -> x ** 2) (map (\y -> sin (y * pi / 10)) [0..9]))

https://www.schoolofhaskell.com/user/commercial/content/vect...

At the risk of going all peak-HN, these things are more a problem with implicit, nonuniform behavior which well-designed languages solve with abstractions like the Copy trait in Rust or even C++ move semantics.

Failed two questions in the first part because I don't understand Python tree syntax and don't know what does it pass by reference and what by value.

Would be great if authors translated it to at least a couple of other languages.

Thanks a lot for that short quizz !

its a python quiz.

Someone please change the title to "15-question python programming quiz". Or python and js? I stopped at the 2nd python "trick" question.

Also, remove the "generalist" reference. Generalist those questions aren't.

Expected to dislike it, but ended up liking it quite a bit.

I admire triplebyte's effort and the content they produce. Their narrative is really data-driven common sense: some engineers fit to some companies but not to others. Especially enjoyed: http://blog.triplebyte.com/who-y-combinator-companies-want

I am a programmer and I run http://www.coderfit.com, a tiny recruiting agency in Zurich / Switzerland (reach out to me via e-mail address in my HN-handle if you look for a tech job in Zurich). Recruiting is really, really hard and I am happy to see platforms like http://interviewing.io and http://triplebyte popping up that give programmers without classic CS background a chance.