What being a rubber duck taught me about problem solving

Long ago, before I discovered the web, (yes, there was a time before the Internet had entered everyone's lives) I was working as a Mechanical Engineer in training.

One day, I get called to the boss' desk (open plan happened before the internet), and he's got this problem. Thousands of steering wheels we manufactured at a plant 600KM away weren't working.
"Not working, how can a steering wheel… not work?" I asked, puzzled.
"Well, they're making a noise" says the boss. "something about a squeaking noise when turning the wheel. They want to send the whole batch back to the plant, scrap the lot, get another batch and/or fine us".

When you're building cars, steering wheels are serious business, I guess. The whole car plant would stop, wait for the new batch, and lose money, because every 11 minutes, a car was rolling out of the doors, to fulfil an order. Everyone up the entire chain was going to get hit.
I had to go to the customer's plant, about 50KM away, and solve the problem. In retrospect, sending a junior trainee wanna-be engineer in training to the plant was probably just an exercise in demonstrating action to the client, whilst they flew the REAL experts in. This, in itself is a clever move, but I digress…

I end up at the plant. Everyone's tense. The customer has decided to keep producing vehicles, in the hope that at least some will be OK, and/or that the problem, when found, could be fixed relatively easily. In the meantime, they've got cars, filling up their buffer parking-lot, out there in the hot African sun, not earning any money. The Lead Engineer in charge of QA takes me out to the lot, and I see something like 200 cars there. All are the luxury, high-end model. Popular seller. Reputation builders. And now, something is wrong…

Giving me a bunch of keys, he tells me to check out any cars I want. They all behave the same. There's a number on a tag attached to each key, and the corresponding number, written in big, angry letters in white chinagraph pencil on the windscreen. He turns, shoulders slumped toward the plant. "I'll be in the QA office" he shouts over his shoulder. He's aware of the theatre here, and doesn't have the energy or time to participate. Likely he's going back to the office to work on his CV or cry or something.

Swearing mentally in all the languages I know, fighting nausea and the urge to run away, I climb in the first car. I've never even been in one before. To this day, I still don't actually own one. Starting it up, it purrs like a kitten. All I can hear is the faint wheeze as I move around in the fancy leather seats, looking for controls and levers. Turning the wheel, I hear a faint (very faint) scratching noise. "If this is the problem, rich folks need to get over themselves" I think. Driving the car forward, starting to do laps around the parking lot, I come up to a left turn.

As I start turning, as the wheel is turning, a noise like fingernails scraping down a blackboard hits me. I'm not going to lie: If I got this car for free, I'd rather rip out the steering wheel and use some pliers to turn the wheel than have to deal with this. It's that bad. "Jesus" I think. "We're hosed".

Every car, same thing. Same level of noise. Whether I'm turning left or right, same awful noise. I can't find any binding between the wheel and the case around the shaft - there could be a defect in either the casing or the wheel, and maybe things are touching and rubbing. This is not the case. Maybe (inconceivably, but still checking) there's a piece of wire or metal shavings or something that haven't been cleaned off the wheel or housing that are rattling around or something in there. Not happening. Absolutely no inspection I make from any angle (on the outside) shows anything that doesn't make sense.

Sweating, I close the door on the tenth or fifteenth car I check. It's been about 2 hours now. Time to head over to the QA office.

Getting back to the office, I find the engineer, not enjoying lunch in the canteen, but with schematics for the steering wheel spread out over his desk, micrometer in hand, checking a handful of wheels.
"Definitely a noise, only when I turn, no obvious cause" I say. "Yup - we knew that 3 days ago, junior" he says with his eyes.

Nothing to do but scrap the batch and try again.

In desperation, I say: "Could you take me through the part of the line where the steering wheel is attached?". A flicker of "Really, are we REALLY going to do this" over the guy's face, and "Sure. Why not" he says, shrugging. It's not like anyone's going anywhere.

We head over to the line. The car is kind of assembled, dashboard and steering housing in place. There's a guy there, taking the wheel, running it through a series of different sub-stations, and then attaching it to he steering column with a great big lock nut. The QA guy starts his patter: School kids come through here on tours every couple of months, and he begins to settle into the rhythm of it.
"First" he says, "the technician attaches the contact plate. This is part of the circuit for the hooter." The tech attaches a galvanised plate using some screws.

"Then" he continues, "the technician attaches the wires going to he other hooter buttons". There's a big central push point, and then two auxiliary buttons on the arms of the wheel. "This allows the driver to hoot using any one of the three places on the wheel" he says, pointing to the places on the wheel where this is possible.

"He then attaches the wires for the contact plate to the pressure plate". The QA guy points to a sort of plate thing with springs on it. It allows the driver to aggressively hit the main, big hoot area in the center of the wheel, forcing the contact plate down, but not crushing it.
The tech then takes the wheel to the half-car, and puts the wheel in place, ready to use the pneumatic tool to attach the lock nut.

The QA guy pauses.

"Peter" he says to the tech. "What's going on? - you're supposed to grease the contact pin".
Peter, half talking over his shoulder says "We ran out about a week ago, but it's fine. The hooter still works". He attaches the wheel, presses the cover for the bolt in place, so the wheel looks like it would when you or I are using it, attaches some crocodile clips to some wires dangling out of the dashboard assembly, and presses the central, then each of the auxiliary buttons. A big red light on a console off to the side of the assembly line blinks, indicating a circuit closing.

As if to cut off any questions about blackboard-nail-sounding-scraping, Peter rocks the wheel back and forth, looking the QA guy in the eye. "No noise" he says, satisfied.

QA guy isn't happy. This is clearly wrong. He tells Peter to come with us, and bring a screwdriver and manual socket wrench. We're going to the yard. "But, the line…" starts Peter. If he leaves, it stops. QA guy smiles. "It's OK, I think we can let things go a bit".

In the yard, we go to one of the cars I've just driven. Peter pulls off the cover, takes the bolt off, and pulls the wheel out. The contact pin has no grease. QA guy pulls a stick of lip balm out of his pocket, gives the pin a few dabs, and Peter reattaches the whole thing. We all climb in the car, and go for a spin. No more noise. Peter is shocked. Somewhere between the floor and the lot, whatever grease from the production process has worn off, leaving metal to rub on metal, causing that awful sound.
QA guy turns to me a bit embarrassed. "Thanks. We've got it from here. Looks like we found the problem. Nice work." He doesn't look too thrilled about it, but his body language is telling me that there's hope: people won't be out of a job anytime soon, they'll get it sorted in a day, and things can carry on as usual.

Years later, in the software development world, with "rubber duck debugging" being a thing, the lesson is simple: rubber duck debugging works, because it forces you to pay very close attention to everything. You have to reexamine your assumptions. You have to start from a place of "rethink everything".

I believe that is probably the single most useful idea to hold on to when trying to figure out what is going on in a piece of code that is misbehaving.

HOW to commit to nothing

In http://ryan-white.blogspot.co.za/2015/10/three-rules-for-sustainable-and.html, I wrote about 3 rules, the third being COMMIT TO NOTHING. In this post, I'm going to talk about how to make this (not) happen ;). Things to think about. Tips and tricks. I could almost write a book about each point here, but I'm not going to put you through that.

1 First: WHY?

Simple. When you commit to nothing, achieving it is easy ;)
Of course, this is a bit tongue in cheek. You're going to have to deliver SOMETHING at some stage, because, well, if you don't, you won't get paid.
"Commit to nothing" really means "Manage expectations VERY carefully".

2 Committing to nothing is NOT THE SAME AS SAYING NO

Devs, a lot of times, have a reputation as the ones who can only see problems. The ones who's first response to anything new or exciting is "NO".
In a world where there's a lot of noise generated when things break, but everyone quickly gets used to everything working 100% fine, this is a natural defence mechanism. But it slows down progress.
Rather than "No, because…", you should aim for "Yes, but…".
"YES - we can do this last minute rush project for you, BUT… which items/projects would you like us to delay instead?".
"YES - I think that idea should be possible, BUT… we're going to need to investigate, to figure out timings and impacts".
"YES - we can talk about this scope change, but would you rather have us talking about the new feature, or finishing the stuff we're scheduled to work on now?".
You get the idea: All work is really a matter of logistics. How many hours of development time available, vs how many are needed to get all the things done. Communicating the difference is vital. This is why Rule#2 - TRACK EVERYTHING is so vital: when you have data on how long things take, you're able to negotiate and estimate with confidence.

3 Time is your friend

Unless you're in a mission critical "production is down" type of incident, there's no need to make decisions or commitments in real time. No work should be undertaken unless the estimations and planning have been done. This doesn't mean that you should be delaying unnecessarily. You should be working as fast as you can to reduce uncertainty and get a reliable, dependable estimate and impact assessment out. Agile needs less certainty and upfront estimating to get started than Waterfall - so the amount of scoping you do depends on requirements of the methodology.
Say "I'll get back to you", and then do that.

4 The people doing the work make the estimates

It doesn't matter if you have consultants telling you how long things SHOULD take. It doesn't matter if you're a team lead, with experience in the tasks, and have an opinion on how long you WOULD take. It doesn't matter if you have the CEO ranting and shouting in front of you telling you how long you MUST take. The only people making that call will be the people actually doing the work. Whether it's by providing the estimate, and sticking to that, or taking whatever time is dictated, and delivering to the time they would have estimated anyway, the team actually doing the work determines the delivery date.
You'll need the entire team to chip in with estimates for their bit. Make CERTAIN the team is in "Estimation Mode" - in a planning session, or similar. When hearing about the concept for the first time, or sitting in a meeting with CLIENT is not the right time. If you're a lone dev, this whole story still applies. You need time away from everyone to think carefully about what needs to be done…

5 Remind everyone that you're committing to nothing. Regularly.

Unless you're constantly reminding people that NOW is not the time in which you're committing, that THIS discussion is only about what MIGHT be possible, but you're not committing to anything, your statements on whether things are possible, or how achievable you THINK things are will be taken as absolute, iron-clad commitment by the CLIENT - who is under intense pressure to get things done.

6 When you DO commit, DELIVER

All of this "I'm committing to nothing now" talk will upset people, and things could get quite heated. You want to make people's problems smaller, not bigger. Introducing "uncertainty" into the situation by repeating the mantra "I commit to nothing" is not going to help calm things down. It is vital that, when you DO commit, you DELIVER. When you first get a request to do something, don't commit to doing it, but do commit to getting back with a first-pass estimate in some near timeframe. Within 24 real-time hours is preferable.
When you send the first pass, make sure to be clear about what you are and aren't committing to yet. Usually, you still need to run some spike solutions and do more research to get to a nice level of certainty. From the first pass, you should be able to give estimates for when you will have the scope planned to a level you're willing to commit to. And so it goes.

7 Conclusion

By being careful about the expectations you create, you increase certainty: building a reputation of dependability, one round of work at a time.

8 Glossary:

• CLIENT: the person who will ultimately say the work is done. In the Agile setting, the Product Owner.

< How to track all the things ...  ... How to deliver >

[edit] - added image

HOW to track all the things

In http://ryan-white.blogspot.co.za/2015/10/three-rules-for-sustainable-and.html, I wrote about 3 rules, the second being TRACK EVERYTHING. In this post, I'm sharing a bit about actual ways to make this happen. Things to think about. Tips and tricks. I could almost write a book about each point here, but I'm not going to put you through that.

1 time, Time, TIME

I can't say enough about time tracking. It's vital for your ability to deliver consistently to be good at estimating, so when you're asked to DELIVER, you can :)
OUTLINE & ESTIMATE: Break down the project into finer and finer pieces, until you have items with a MAX of 4 hours estimated effort. It's always about time. I know, I know, Story Points are about complexity. But, inevitably, you end up talking about Velocity - which is really about points per Sprint, which is a time-box. Your estimates are best "Naive". How long it'd take you to do this bit of work, if you were working on just this task, and weren't experiencing interruptions. Time to execute, not actual time for this to be done. So it excludes time spent waiting for other people to check/code reviews/budget approvals or whatever. If you KNOW you're going to need a code review, that's a separate task, with its own estimate.
TRACK ACTUAL: The point? - you're trying to see how accurate you are when estimating. Track your actual time taken, and at the end of the week/sprint/project/whatever, have a look at your estimated vs actual times. Learn from this, see if you can figure out ways to bring the two figures closer together. It also doesn't hurt to be able to say (and back it up with figures) that you track every minute of your day.

2 Master the tracking systems you're given

Most companies have some sort of timesheet system they subject the workforce to. Understand that this isn't going away. It is to your ADVANTAGE to become a pro at it. I use Emacs org-mode, and then funnel data from THAT into whatever system HR is running. The reason? usually, the HR system can't help with planning or estimating. With org-mode, I can outline, estimate, track and report, all from one system, at a much richer level than any other system I've encountered. And I'm in full control of the original data, so it won't get lost or mangled. And I'm able to process and query it for interesting insights.

3 Track conversations with writing

After any meeting, deskside chat or watercooler session in which a decision or agreement is made, back it up with a mail. Or other written medium - as long as you can go back to it later. You can check on it and remind yourself what to do (so not wasting anyone's time) - and in the event of disagreements/arguments, you'll have something to refer to. It sounds stupid, assuming everything could turn into a persecution session, but remember: TECHNICALLY CORRECT IS THE BEST CORRECT.
Make default choices/assumptions. And share the choices in writing. This helps move things forward faster, and you're minimising the risk of people having nasty surprises by letting them know beforehand what you'll be doing. Of course, only do this when you have run out of other options, such as reading the briefs, chatting/mailing/texting whomever you need advice from etc.
Need an answer, decision or output from someone? Feel free to do so in person, over the phone, or in the hallway. But make sure you back this up with a mail or other long-term written record.

4 Use Analytics

TRACK EVERYTHING applies to your apps/websites too. It's vital for you to know and understand what's going on inside them. Whether you use New Relic or Google Analytics, make sure you're able to see what's happening in your app/site. Mostly, you're instrumenting for the reasons you CAN'T think of. A bug will emerge, and you'll want historical data. Your site will get hit with a ton of traffic, and you'll want to see if there were patterns to people's behaviour. That sort of thing. Often, analysis is done, and people say "wouldn't it be nice if we had xxx data or yyy detail?". Do your level best to make it so this is never said of the work you do.

5 Track your progress with notes/documentation

Coding is difficult enough. Remembering state for the myriad projects you're on is crazy. Do yourself a favour, make notes, and have a 5 minute break. It doesn't matter whether you make the notes in the source code, some text buffer or post-its - the important thing is to do it.

6 Rather over communicate than under

Keeping CLIENT calm whilst you're solving their problems is half the battle. Remember: they aren't telepathically plumbed in to what you're seeing, and the progress you're making. So it's vital that you give regular updates. How regular depends on the client and what you've agreed upon as well as how urgent the job/task is. Even if you have nothing to say, other than "I'm still investigating" - let people know you're working on their stuff. Don't lie. If you're only going to get to their stuff later, that's OK, just let them know.

7 Bonus: WHY to track all the things

Simple, really. When you've made an audit trail of what you're doing, you don't have to remember trivial details, like when you started and stopped a task, and what you were doing last Tuesday. If you're a contractor, you're already focused on this, either because you're billing by the hour, or have committed to a fixed-cost project, which is based on estimated hours… If you're permanent, remember: you've signed an agreement for x many of your hours per day. It is optimal to work that number of hours per day. Any more, and you're being exploited. Any less, and you're stealing ;)

< How to deliver ...  ... How to commit to nothing >

[edit] - added image

HOW to Deliver

In http://ryan-white.blogspot.co.za/2015/10/three-rules-for-sustainable-and.html, I wrote about 3 rules, the first being DELIVER. In this post, I'm sharing a bit about actual ways to make this happen. Things to think about. Tips and tricks. I could almost write a book about each point here, but I'm not going to put you through that.
Some useful tips for delivering

1 What's the actual problem here?

Often, you're told to do X to fix Y problem. Spend a couple of minutes figuring out if Y is really a problem, and if there isn't some better/easier way than X to fix it. I've saved countless hours with this.

2 What can be done RIGHT NOW, and then what is the correct solution?

There's always a problem with time. Figure out the absolute bare minimum that needs to be done RIGHT NOW to solve the problem. Then think about how to solve correctly. You might only have time to figure out the RIGHT NOW solution, but you'll be aware that it's going to need attention later on.

3 Naive estimate first, THEN optimise.

Don't waste mental cycles and time trying to shoe-horn everything into an elegant, efficient model or schedule from the start. Treat each part as if you were doing only that part. As the scope becomes more concrete, you can optimise for re-use, common functionality etc.

4 Figure out what needs to be done first

Sounds obvious, but your predisposition for linear thought is acting against you here. You need to weigh Urgency vs Importance vs Difficulty vs "Force Multiplication" vs "Internal Setup Time". Using these, you'll end up with a sometimes non-obvious order of execution that is better.

5 Sub 4 hour resolution

Aim to have no task estimated at longer than 4 hours to complete. This seems like overkill, but trust me - paying this much attention when estimating means you're much less likely to overlook things. And you end up with a plan that can tell you, within half a day, whether you're ahead or behind schedule.

6 Spike solutions

For things that are unknown, or difficult, use a spike solution. Zoom in on the specific technical problem you're trying to solve, figure out the smallest possible thing you have to do to solve the problem, and solve just that problem. The point is not to solve the larger problem: it's to help you understand unknowns, so you can better plan your way forward.

7 Solve end-to-end, then polish.

An inefficient, badly written complete solution is better than the most elegantly written incomplete solution. Get something working that solves the problem as quickly as possible, then focus on parts that need refactoring or rework. Unless you're building an Air Traffic Control System. Then get it right the first time.

8 Notes, for start-stop work

Do yourself a favour: keep yourself well informed. Never assume you're going to remember everything. Write notes down (either as comments in the code, or separately) as you go.

9 Work in bursts

I typically never work for more than 50 minutes without taking a break. Often, taking a break is just getting a cup of coffee, going to the loo, or reading an article for 10 minutes. Work hard, and reward yourself with little breaks. Most of the time, better approaches or insights happen when I'm NOT at my desk, thrashing out code…

10 Remove distractions

Plan for "available time" and "unavailable time". Disable notifications as much as possible. Check your mail once every hour or two only, and schedule actions from the mails, rather than trying to deal with them on the spot. Work from home. Find an isolated spot at the office somewhere.

11 Multiply force

Have sections of work that others are waiting for you to do, before they can move on? Kill those ASAP. Try never to have people waiting for you.

12 AUTOMATE

macros/scripts/applications/other people - "Anything worth doing twice is worth automating" is maybe taking it a bit far, but repetitive tasks are best done by machine or multiple other people - you're just being a bottleneck otherwise.

13 Glossary

• Spike solution: notes here: http://www.extremeprogramming.org/rules/spike.html
• Urgency: Anything that must be done NOW - that's holding up other people, or preventing people from using your software correctly, or closing a lead/sale is Urgent.
• Important: Anything that will have a big impact on your business/unit/app/site in future. Often expressed as "let's worry about that later" by the suits, until it becomes Urgent.
• Difficult: Anything that you can't get your head around in an hour or so. When you answer a question of whether something can be done, and you initially say "no" - it's likely difficult.
• Force Multiplier: https://en.wikipedia.org/wiki/Force_multiplication explains well, but basically, if you can build a tool, app or process that allows many other people to do work on your behalf, or get it done automatically, that's a force multiplier.
• Internal Setup Time: "Internal setup activities are those that require the process to be at a standstill before you can conduct them safely while external activities can be done while the process is still running." (check out http://setupreductiononline.com/ for the full story). Yes - the roots are in the changeover of press dies, but there are many parallels and lessons to be learned for software engineering. If you can find a way to reduce internal setup time, the process of making something new or fixing/repairing happens much faster.

< How to commit to nothing...  ... How to track all the things >

[edit] - added image