Re: Ungooglable interview questions
On 17/03/13 09:54, Sam Kington wrote: At $WORK we ask new hires, over IRC (because everyone potentially works remotely), a variant of chromatic's http://modernperlbooks.com/mt/2011/01/how-to-identify-a-good-perl-programmer.html So far, when we've put people through the test (some people have been omitted because they obviously had a clue), nobody has managed to identify * as a sigil. I'm slightly concerned that the specific questions are susceptible to being googled. Does anyone have any good examples of non-googlable interview questions that could be answered (or not) over IRC or a similar medium? The usual standby, asking people to write code during their interview, obviously doesn't work when someone can google, cut and paste. Sam You could ask your questions over Google docs or some similar system that shows you real time what the interviewee is typing. If you suddenly see 100 lines of code appear you are either dealing with the world's fastest typer or a copy-and-paster. Skype also has a share screen function. Good when you have both have a good internet connection. AJ
Re: Ungooglable interview questions (was: Re: New perl features?)
On 17 Mar 2013, at 09:24, Abigail wrote: > On Sun, Mar 17, 2013 at 01:54:37AM +, Sam Kington wrote: >> At $WORK we ask new hires, over IRC (because everyone potentially works >> remotely), a variant of chromatic's >> http://modernperlbooks.com/mt/2011/01/how-to-identify-a-good-perl-programmer.html >> >> So far, when we've put people through the test (some people have been >> omitted because they obviously had a clue), nobody has managed to identify * >> as a sigil. > > Have you hired any of the people any of those people? If so, does this > question have any value? Does it really matter when someone during an > interview can remember that * can be used as a sigil? As I said, we haven't made obviously clueful people go through this test, and those are the people I'd expect to remember "ahah, yes, there's * as well". But the rest of the question is potentially useful for identifying how comfortable with the language people are, and leads nicely into questions about hash slices. It's a decent shibboleth, in other words. > But loooking at chromatics list, does it actually work? Does it identify > good Perl propgrammers? I guess it can be used to weed out people claiming > to know Perl very well, but don't, but someone who can answer all the > questions may still not be able to code his/her way out of a wet paper bag. It's a good quick, basic, test of how good at the language people are - in half an hour to an hour you can get some general idea of where someone falls on the scale of ignoramus -> junior developer -> potential senior developer. I'd still want to see someone's code, and have them talk about past projects and how they went around solving problems, to get an idea of how the person's brain works, especially if hiring for a senior position. Sam -- Website: http://www.illuminated.co.uk/
Re: Ungooglable interview questions (was: Re: New perl features?)
On 17/03/2013, at 8:24 PM, Abigail wrote: > On Sun, Mar 17, 2013 at 01:54:37AM +, Sam Kington wrote: >> >> At $WORK we ask new hires, over IRC (because everyone potentially works >> remotely), a variant of chromatic's >> http://modernperlbooks.com/mt/2011/01/how-to-identify-a-good-perl-programmer.html >> >> So far, when we've put people through the test (some people have been >> omitted because they obviously had a clue), nobody has managed to identify * >> as a sigil. > > Have you hired any of the people any of those people? If so, does this > question have any value? Does it really matter when someone during an > interview can remember that * can be used as a sigil? > > But loooking at chromatics list, does it actually work? Does it identify > good Perl propgrammers? I guess it can be used to weed out people claiming > to know Perl very well, but don't, but someone who can answer all the > questions may still not be able to code his/her way out of a wet paper bag. > I''m not sure I'd want to hire someone for a routine job who was keen to remember that * is a sigil. If someone I know and trust uses it, then it's usually clear what they're doing with it without having to talk to them about it. OTOH I doubt I'd want to hire a random off the street who was keen to tell me about it unless we started off the discussion with an advisory about chicken's blood and mystical symbols, and that the rest of the discussion was about more conventional approaches. Also, effective use of search is a desirable attribute in a technical person. I'm not sure I'd be keen on an un-googleable trick question. I think that the correct approach is to triangulate such that you have multiple independent lines of evidence that the person you have been talking to is right for what you're offering.
Re: Ungooglable interview questions (was: Re: New perl features?)
On Sun, Mar 17, 2013 at 01:54:37AM +, Sam Kington wrote: > > At $WORK we ask new hires, over IRC (because everyone potentially works > remotely), a variant of chromatic's > http://modernperlbooks.com/mt/2011/01/how-to-identify-a-good-perl-programmer.html > > So far, when we've put people through the test (some people have been omitted > because they obviously had a clue), nobody has managed to identify * as a > sigil. Have you hired any of the people any of those people? If so, does this question have any value? Does it really matter when someone during an interview can remember that * can be used as a sigil? But loooking at chromatics list, does it actually work? Does it identify good Perl propgrammers? I guess it can be used to weed out people claiming to know Perl very well, but don't, but someone who can answer all the questions may still not be able to code his/her way out of a wet paper bag. Abigail
Re: Ungooglable interview questions (was: Re: New perl features?)
On Sat, Mar 16, 2013 at 6:54 PM, Sam Kington wrote:
> I'm slightly concerned that the specific questions are susceptible to being
> googled. Does anyone have any good examples of non-googlable interview
> questions that could be answered (or not) over IRC or a similar medium? The
> usual standby, asking people to write code during their interview, obviously
> doesn't work when someone can google, cut and paste.
In practice it's probably too hard to copy & paste code content for an
interview.
Even if it weren't it's easy for you as an employer to ask questions
about said code to test understanding. Everything from "explain the
syntax of that declaration" to "explain your choice of top level
algorithm" to "what other ways could you achieve this small
aspect/large aspect/entire thing" plus you can always insist they TDD
the solution; that happened to me recently for a ruby gig and so I
wrote a nano-test framework since I couldn't remember the APIs of
anything outside of rspec :D
Going further you can start to ask variations like the same program
with different inputs, outputs, performance characteristics (running
on VM v. metal), different environments ("your network is really
laggy, how would that affect it? How would you mitigate?"), different
languages ("what programming languages have you used that might be
more suited to this & why? any that would be awful for it?")
I've seen a colleague who had a fake question that had an answer
online he'd written to catch people searching for the question, so
there's that too ;-)
Paul
Ungooglable interview questions (was: Re: New perl features?)
On 16 Mar 2013, at 14:48, Dave Cross wrote: > On 16/03/13 11:42, DAVID HODGKINSON wrote: >> On 16 Mar 2013, at 08:31, Dave Cross wrote: >> On 03/15/2013 10:40 PM, DAVID HODGKINSON wrote: >>>> ~~ and //= are the only new sigils >>> >>> I don't think 'sigils' means what you seem to think it means :-) >> >> Line noise? > > They are operators. Sigils are the things that go in front of variable names > - $, @, %, & and *. No new sigils for a long time. At $WORK we ask new hires, over IRC (because everyone potentially works remotely), a variant of chromatic's http://modernperlbooks.com/mt/2011/01/how-to-identify-a-good-perl-programmer.html So far, when we've put people through the test (some people have been omitted because they obviously had a clue), nobody has managed to identify * as a sigil. I'm slightly concerned that the specific questions are susceptible to being googled. Does anyone have any good examples of non-googlable interview questions that could be answered (or not) over IRC or a similar medium? The usual standby, asking people to write code during their interview, obviously doesn't work when someone can google, cut and paste. Sam -- Website: http://www.illuminated.co.uk/
Re: Interview Questions
On 09/05/2012 06:34 PM, Simon Wistow wrote: On Thu, Sep 06, 2012 at 12:12:16AM +0200, Joel Bernstein said: Really I find interviews are less about individual questions or tasks, and more about where the conversation goes based on them and the interviewer's abilities to steer the conversation and to assess the candidate based on those discussions Yes. Yes. Thrice yes. and i have said that here a few times already. it is also the core tip i give to my candidates when they are interviewing. it is not an interrogation but a conversation. you should ask questions and tell stories about your projects. this is closer to a real work situation where you are discussing how to solve a project. uri
Re: Interview Questions (was: Brainbench)
On Thu, Sep 06, 2012 at 12:12:16AM +0200, Joel Bernstein said: > Really I find interviews are less about individual questions or tasks, > and more about where the conversation goes based on them and the > interviewer's abilities to steer the conversation and to assess the > candidate based on those discussions Yes. Yes. Thrice yes.
Re: Interview Questions (was: Brainbench)
ok now that the subject line has changed to reflect the interview process, here is what i think the ideal interview is (experienced this both as interviewee and as hiring manager.) it should be a very short internship, i.e. here is a masked part of a problem we are working on, lets discuss this for a bit, then i'll let you work on this 2 hours and we'll meet again to discuss. when I interviewed (for another company some time ago) the entire algo team showed up for the post work review, and this happened twice, it was a 7 hour day. (~3.5 hours per problem) when i interviewed , i started with a homework question and then discussed it in the interview. (and asked followups) nothing like seeing someone do actual work, while looking over his shoulder on an actual computer. here, this is the entry point, have a look. http://upstream.shalit.name/ -- vish On 5 September 2012 23:12, Joel Bernstein wrote: > On 5 September 2012 23:15, Daniel Perrett > wrote: > > The fib(n) task as stated tests three things of the candidate > > You're missing a key point of interviewing. The task is used to test > how the candidate thinks, how they ask questions, how they explore a > problem space etc. It has to be discussed and interpreted - simply > marking an emailed submission of code to solve fib(n) would be > unhelpful. > > As an interviewer your job is to measure their abilities/deficits with > regard to understanding and exploring new subjects, at least as they > relate to the kind of work you'll be asking them to do. It's not even > just about assuring yourself that they're capable of doing the job, > you need to be sure it's a job that will satisfy and interest them. > I've never set anybody the fib() test but I find it very enlightening > to work through a programming task with a prospective candidate. > > Really I find interviews are less about individual questions or tasks, > and more about where the conversation goes based on them, and the > interviewer's abilities to steer the conversation and to assess the > candidate based on those discussions. Which is one reason why > pre-interview screening tests for developers only go so far. It is > arguable of course that my methods may select for more articulate > programmers but I do try to ensure that the interviews are relaxed and > friendly. If a capable engineer can't communicate in those > circumstances there's always a doubt about how much they will be able > to offer in a team anyway. > > /joel >
Re: Interview Questions (was: Brainbench)
On 5 September 2012 23:15, Daniel Perrett wrote: > The fib(n) task as stated tests three things of the candidate You're missing a key point of interviewing. The task is used to test how the candidate thinks, how they ask questions, how they explore a problem space etc. It has to be discussed and interpreted - simply marking an emailed submission of code to solve fib(n) would be unhelpful. As an interviewer your job is to measure their abilities/deficits with regard to understanding and exploring new subjects, at least as they relate to the kind of work you'll be asking them to do. It's not even just about assuring yourself that they're capable of doing the job, you need to be sure it's a job that will satisfy and interest them. I've never set anybody the fib() test but I find it very enlightening to work through a programming task with a prospective candidate. Really I find interviews are less about individual questions or tasks, and more about where the conversation goes based on them, and the interviewer's abilities to steer the conversation and to assess the candidate based on those discussions. Which is one reason why pre-interview screening tests for developers only go so far. It is arguable of course that my methods may select for more articulate programmers but I do try to ensure that the interviews are relaxed and friendly. If a capable engineer can't communicate in those circumstances there's always a doubt about how much they will be able to offer in a team anyway. /joel
Interview Questions (was: Brainbench)
"Given that fib(n) is equal to fib(n-1) + fib(n-2) write a fib function in any language" The fib(n) task as stated tests three things of the candidate: - Are you confident enough to consider that the interviewer might be presenting a problem as complete which in fact requires some more info? (And to risk asking?) - Can you spot the trick in the question? - Can you perform the fully specified programming tasks, once you've spotted the trick? While I wouldn't say those are invalid, the question for you, the interviewer, is "do they tell you what you need to know?"; the answer may depend a lot on whether you're after someone you are expecting to train up or someone you expect to be refactoring your legacy code in their first week. I'm going to assume something closer to the former. The confidence issue is a big question and I'd argue you need to mitigate it. It's going to depend on all sorts of factors including how stressed they are (if it's their ideal job, the answer is *very*), how many people you've interviewed (i.e. how impatient you seem), and whether it's before or after lunch. Whatever problem you set, you should do make it clear you expect people to ask questions (even dumb ones). Sure, hubris is a Programmers' Virtue and a good employee will be confidently asking tricky questions and poking holes in the spec, but compared to the interview room that's a very different social relationship. Confidence (natural, socialised, or situational) gives you an advantage in interviews anyway, and I think it's important to avoid increasing that advantage. "Spot the trick" can be a useful skill, in fact, you can argue it's similar to the skills we use for debugging, but it's impossible to gauge reliably based on one problem (especially when the problem is equivalent to a well-known one which people might know by chance rather than because they figured out you need the additional info). And because the problem is mathematical, people with academic mathematics-like backgrounds will see that quicker, and I suspect that's going to end up selecting for a less diverse range of experiences/ideas/approaches/demographics than you'd like (but hey, maybe you feel strongly linguists will mess up your Perl code, I dunno). If you want to do "Spot the trick", avoid known sequences. But because you don't want people to focus on trying to guess if it's ok to question the question in an interview, I would simply recommend saying 'Here is a function to do X, it doesn't work: why not/what does it do instead?'. Performing defined programming tasks is something you might want to test, but you could do a selection of simple and clear tasks in a pre-interview question without the trick, It's a filtering question. If you're doing this, I would recommend something which people can picture rather than just "numbers", e.g. triangle numbers, as this minimises the chances that you're explaining things less than clearly or in a way inaccessible to the candidate, or in a way which makes them pore over the wording looking for tricks. (I know I just said avoid defined sequences but that was if you were testing something else; I am suggesting you choose one or the other). Things it doesn't necessarily test for, but which you might be interested in: - Is the candidate able to grasp and apply the principles of recursion, even if they've not used it before in a language? - Can the candidate turn an iterative function into a recursive one? - Does the candidate know (or can they meaningfully discuss) when it is best to use recursion? I'm writing this not because I think it's heresy to use this task, but to use it as an example to think through why we do tasks in recruitment. We need to think hard about and talk a lot about these questions, because we are Perl devs and not HR experts. So I guess as a positive recommendation: rather than a simple "is problem X good or bad?", the question we should be asking ourselves is "I want someone who is capable of doing A/B/C, and I am also interested about what they know about X/Y/Z. How do I design and present my problems so that as far as possible they identify all those and only those who can do A/B/C and give them the opportunity to demonstrate their knowledge of X/Y/Z?" Daniel
