Lessons Learned: Reflections on a recent interview

This post was moved from LinkedIn (now owned by Microsoft) to this Blogger site (owned by Google) because I was concerned about potential censorship (see Update regarding my mentioning here about doing a Glassdoor review) despite the fact it is simply a wish list of features that were my own ideas, inspired by a recent interview. So the move is may be unnecessary, hopefully...

I've hesitated to do any posts on LinkedIn for many years - more than sharing updates, I mean. Why? Well, I wanted to have something relevant to say and wanted to do it in a way that wouldn't be seen as "simply just another rant."

But I was inspired recently after a PM interview at Microsoft, not the interview itself, but rather some of the implications involved. Now, before anyone starts jumping on me saying "but the NDA you signed...!" let me stress that the NDA you sign covers not disclosing confidential information the company (via its employees or papers, etc.) may reveal to you about the company or one of their partners, and that the information conveyed could not have been gotten via a public avenue. It does not cover stuff I told them or features I suggested to them as part of the interview.

That's one of the gotchas with interviews, all types - sometimes it feels like the company is simply trying to get free consulting or free work (as with artists being asked to do 'spec work' or localization experts asked to translate something that is obviously real translation work or data people being asked to "run some numbers"). We all know or have heard about "coding tests" and partly because of that notoriety most coding tests are designed to be firmly "not for actual use in the company's product." Well, not so with this PM interview, which was about taking you throughout a typical feature/issue problem with the actual product, working out scope, customer insight issues, hypothesis and ways to split test those hypothesis and above all, coming up with features to fix the issue and make the product better. It's actually a lovely way to conduct such an interview, working through a problem with colleagues, them seeing how you work and think - but there is a downside when it's based on the actual product, for both you and them.

If you're conscientious, there is almost no way to avoid doing real "free" work for the company in that kind of situation, since you are making real product feature recommendations and working out how to test whether they solve the product's focal issue. You could say "Hey folks, this has crossed the line into consulting, so either you sign a form saying you won't use these feature recommendations or you pay me my consulting rate from this point forward..." Or you can try to avoid mentioning any features, except they will ask you directly for them or think you don't have any! You're in a box because you know if you do any of the those things, the interview is over and you'll be the goat.

So you go along with it. If you're lucky, they'll hire you and all well and good, but what if you're one of the majority that don't get hired? It definitely affects you. And despite the fact that you know that most interviews you go on will result in rejection, it doesn't really help. (I saw one person quote, based on his own experiences, that only about 10% of the job interviews you go on will result in an offer...pretty dismal, and likely variable in many ways, but in a competitive market - even in tech, and especially if you are not the ideal stereotype for the job - it's likely close to the truth.)

So rather than get upset or depressed, I simply added my experience to Glassdoor, noting how often they said "Wow, that's a really good idea!" which was then followed by them noting it, writing or typing it depending upon their chosen path of paper or tablet. Glassdoor is a great resource, as much as many recruiters and companies hate it*. But then I decided, "Hey, why not also talk about the features I suggested?" That way, they become public knowledge, with my blessing, and free to anyone to accept or discard. If it helps make a product better, then great, and this way Android Studio, Eclipse or anyone else has the same chance to use or discard.

*Update: apparently, like many others who have done neutral or negative reviews (mine was neutral and revealed no confidential info) that come to the attention of the company, I just got a notice from Glassdoor that my review (previously approved) was now rejected. And when I went to update it, per their instructions...I found my account had been deactivated so that it would be impossible for me to do so. Very sad to learn that. So now LinkedIn is a concern as well. Very disappointing. I learned also that it's become a common practice at Glassdoor - which is even more disappointing considering how they claim that no employer is allowed to interfere. I hope this type of behavior will change and only spam or breaches of confidentiality will be dealt with, not honest feedback meant to help others (and the company, if you think about it.) Transparency is important, especially during these chaotic times.

But that just means that Glassdoor (and perhaps eventually, LinkedIn) may lose favor among job seekers (and therefore recruiters) and some new sites will become the trusted sites.

So here's the list: The product being discussed was Visual Studio, the IDE in general and C++ in particular, and the problem was how to increase customer engagement with a product that is very large with lots of capabilities but as a result, very slow to install. I was the one who brought up the install time as key - so they didn't ask for that. It was my choice to focus on that aspect of the product.

Since I've done a lot of research on 'engagement', I redefined their use of it (they were using it more like 'usage'), to the definition I came up with. Engagement is high intrinsic motivation plus high interest. What does that mean? High interest just means what you think it means, you want to learn more about X. High intrinsic motivation means you are willing to learn more about X and will learn more about X, and you don't need any external motivations like your boss telling you that you have to learn it, or being paid to learn it, or getting a grade to learn it. It just makes you feel better about yourself when you learn about it, and perhaps makes you feel more masterful and competent, or even that your status will increase in some way.

So with that in mind, and given that most IDEs comparable to Visual Studio in power and scope would have similar problems, I came up with a number of features that might help improve the customer experience with installation and adoption.

• Use wizards to help guide people in what they want to install versus what the need to install to get up and running as quickly as possible. Let them choose both but only install the 'need' group first and install the 'want' group in the background after launching the IDE. This is coupled with given them time estimates based on what they are choosing so that they can make an informed decision. Have tightly synced walkthrough introduction tutorials and ask the installer if they want to immediately start a walkthrough, knowing that many people will just watch a tutorial first, rather than watch-and-do the tutorial. So the product can be installing while the user gets to know more about the product.
• Have live links in the IDE that keep track of where the user is in his/her learning. In other words, have links back to the tutorials since users will often pause and come back to tutorials. This way the product is keeping track of where they are. To make it even better, keep track of where in a video (or page location) the user last was, so that they can continue directly where they left off.
• Have 'help bubbles' that pop up to help the user learn the new IDE, even stepping them through basic tasks. This is similar to how games help users move through the games with hints (such as an arrow pointing to some place they might want to click or go). Maya has used this kind of hinting system to great effect in past versions. The user also needs to be able to turn it off (easily!).
• Fun means Engagement! Make the tutorials more fun with a real payoff, like Unity does so successfully with their bootcamps, with live help in a forum related to that walkthrough. Intros are just that, about making people feel comfortable, knowledgeable and empowered with the IDE/product. They need to be fun. Yes, fun! It's a sure sign of engagement. Advanced tutorials can be more specific, just as advanced samples should be seen as base code for the user's real code - i.e. not simple, throw-away code but an actual base for their product, both Unity and Unreal do a great job at this. Also, don't have a talking head in a video, especially as the first frame - all you see is a person's face with that big "Play" icon on top of it. Think more about what information you are about to present - and make the first frame a teaser of what the user will be accomplishing or getting.
• Improve startup time for the IDE. I use game engines as my main app building mechanism. I only use it for making DLLs and shared object libraries and won't use it for Unity since it takes so long to start up (compared with MonoDevelop). I thought of using Visual Studio Code, their offering to compete in the web dev space mainly (aka against IDEs like Atom), but VSCode only looks at one file at a time, and I work with clusters of files (either in Unity with many scripts that are all part of a project, or in Visual Studio with its project structure), so not being able to reference and easily find the other files in my project makes it unusable in my situation.
• Finally, though I had quite a few other features ideas, there is one that is very specific to Video Studio as it is a cross platform build system now. Right now, I love Visual Studio for making it much easier to build cross platform, as it also takes care of installing and managing disparate kits that otherwise I would have to wrangle myself (and try to get to work correctly together). It's part of the Python saying "batteries included" where all dependencies are loaded together, simplifying the entire build environment. Still, improvements can certainly be made to any IDE trying to make cross platform easier. So. Keep working on making cross platform building even easier, from being able to make changes to Build settings and saving them for a different Build type within the IDE (right now you have to open up saved build settings for projects in a text editor, in VS2015 you can't just tweak something then say "save as new", you have to create a blank slate and redo every setting the way you want. (Note: I'm using generic terms here, like Build settings.) When you have a lot of variations, like for different platforms or versions, it becomes a pain, and a potential nightmare of errors. Likewise, fix the linker in Android so that the order of libraries doesn't matter. The order of libraries doesn't matter to the VS linker for Windows builds, yet with the linker they are using for the cross platform Android build, if you get the order wrong, you get linker errors (and without any real idea of why you are getting them.) Little things like that go a long way in making a developer's life a lot easier.
• Finally, one other thing I brought up, that I want to mention, was something that I knew wasn't Microsoft's fault, or any companies for that matter, it's the problem of DLL Hell with STL. Those in development, and do cross platform shared object or dynamic library work, will understand that this is a problem with the standard template library and being able to share templates across shared object/DLL boundaries. It's still apparently a problem with C++17, which is disappointing. But again, with coding becoming more and more cross-platform in a mobile first world, solving this problem would be a big win for anyone and everyone. So I'm just throwing it out there.

If you didn't understand some of what I discussed, don't worry, much of it was specific to IDE based features, though much of it can be applied to other software products, or even other products in general. And it demonstrates, in a very simple scenario, that products, even simple games, can be complex in terms of customer insight and making intuitive designs that users will enjoy and want. It's a process and it takes time, but it also shows that if you have high customer engagement, your product has a better chance of success since customers are more vested in its future, and willing to forgive you when you make mistakes. They become your partners in many ways, and you never want to break that trust. So it's good that companies making IDEs are thinking more about engagement and other items often relegated to marketing, usability or even academia.

In the coming weeks, I'll be doing more lessons learned articles, including a breakdown of my first augmented reality test app (for AR for books) and how I'm using those lessons in my new AR engine and test app for books, along with issues with building my own AR engine (including DLL Hell with STL as it relates to OpenCV). I'll also be talking more about data science and customer insight, including reflections and progress on some side projects of mine, as well as issues facing women in STEM, including a book publishing saying that I came across all too often. This saying was used to explain why they would focus their time and energy on books for boys rather than on their larger readership of girls, namely, "it doesn't matter, girls will read anything, but boys won't read about anything but boys." To me, that's a problem, especially as the research I've done shows that it has a negative impact on girl's perception of their worth and the future place in society. It was one of the reason I began looking into AR for books - a way to counteract many of the negative messages girls and other cultural minority groups face every day.