Making software production worthy

Your app, your product, your company’s future. The idea is sound, the market is ready, the team spent many months implementing it.

The questions corporate sponsors will be asking are: is it ready for production? When do we launch?

As an architect, how do you assess production readiness and chances of success in the field?

Here are some criteria to help formulate the answer to that question.

Code quality is good-to-great

Your code has little tech debt. The architecture is flexible, scalable. Features are easy to modify, augment, and add.

The dependencies are at the latest stable version.

The integration points have corresponding integration tests.

Core components have unit tests covering happy paths and failure paths.

Concerns are clearly separated.

Feature set is mature

The team and Product Owner has painstakingly iterated through the feature set multiple times, polishing workflows, interactions, screens, verbiage. Your features don’t just “work well”, the team starts using words like elegant, fun, cool to describe various things the product can do.

The last few times your Product Owner requested a new feature or a drastic change to an existing one, the team did not cringe and ask for “at least two sprints” to deliver it.

Change becomes fun when the software is architected well.

Onboarding, user management, and off boarding works near flawlessly

Once you are in production, you will get real customers. There is nothing more infuriating than an inferior or faulty onboarding process.

Your emails must look fun and exciting. Limit the number of steps until a customer can get in and feel like the registration is over. Defer all secondary actions to post-registration, if at all feasible.

Make it easy and fast for someone to provide the info you need to operate.

Give people options, for example make it possible to log in with email, phone number or user name. If you collect this data and require it to be unique but then don’t let the user identify himself to your system using the data he provided, you are not facilitating the best experience you can. Aside from that, people like to compartmentalize ids and passwords. Give your users the option to put your service in the “I use my phone number for this” compartment.

Change password, log out everywhere, change phone number, add another phone or email, recover password or user name, resend email confirmation – you remembered all of those, right?

Sometimes customers will want to leave your service. I know, crazy… But seriously, if they enter any data into your software, they are likely to want to take it with them and use it with a different service. Think of this and build a data export process before you go live.

Release pipeline is designed, implemented, and tested

Once you hit prod and get real customers, they are likely to be less forgiving and more inventive at breaking things than the best PO. You will have to roll releases fast and furious and your release pipeline better be ready for it.

Test your release pipeline like you test your software. Time it from the request for a code change to commit to build to verification to sign-offs to deploy. Identify the bottlenecks, understaffed areas, weak links, under-automated workflows. Correct the entire pipeline until you know you can release quickly and twenty people don’t have to spend six hours on a phone bridge yawning and hating your software.

Support process defined and tested

Go ahead and call or email your support pretending to be a customer. You are likely to realize that the support process is not ready, does not have enough support scripts, documentation, knowledge of the system.

This is what your customers will go through. Fix it. Document common workflows, escalation routes, build support utilities to help resolve common issues.

Also, do you have a good process for support reps to report frequently occurring issues to devs? It does not have to be catastrophic to become a sprint item, right?

It’s a great practice for support to publish a weekly newsletter with a name like “From the Trenches”, where they report successes, failures, funny stories, etc. We are social creatures, community building is important.

Code and environments are instrumented for analytics, logging, troubleshooting

Instrument your code well. Log important events. Provide workflow traceability. Your team’s post-release quality of life depends on it.

Understand what a customer will provide when calling about an issue and make sure your logs allow you to get from that input to that user’s interactions with your system. For example, customer will provide the email address. Can that tell you the APIs calls and workflows that user was executing and where things went off-rails?

DevOps are mature

As the system matures in production, it is likely to change. The moving parts may change, the hosting strategy may change, the external integrations and corresponding firewall and networking may change.

Is your DevOps organization mature enough to implement and support your release pipeline and rapid infrastructural changes you will throw at them?

To my mind, production-ready level of maturity for DevOps is when they can accommodate networking, security, hosting, pipeline changes within a couple of sprints at most.

How many things are DevOps still doing manually? Try to influence for more automation.

Hosting concerns resolved – certificates, servers, licenses, DSN, firewalls, load balancers, networking, CDN

I can’t tell you how many times I have seen releases plagued by incorrect or insufficient certs.

Weak servers, unpaid licenses, misconfigured DNS and firewalls… I’ve seen and lived through it all.

Test it, drive your system’s physical and logical hosting design, know the full checklist and people responsible for each part. Even if it is “another department’s fault”, it will be your software that won’t work.

Customers never report faulty firewall configurations, they report “I can’t log into your *&#@* app”.

Feedback pipeline

Just like onboarding and customer service, giving your users a well-designed feedback pipeline will make them feel valued and will make your product look professional and polished.

If customers can recommend features, vote for favorite items on your roadmap, report frustrations, your backlog will become a ton more meaningful. And so will your product. Don’t make it an afterthought. Build it before launch. Early life is when you will need it the most.

Beta testers identified, relationship established

Identify early adopters, nurture those relationships, incentivize them if possible.

An active community of early adopters can make the difference between a successful launch and total flop.

Product Owner is happy

Is your PO happy? Is there an excited ringing to his voice when the product is discussed? Is he using it every day? Has he switched from talking about bugs and imperfections to planning for future features and updates?

If your Product Owner is happy, chances are your users will be as well.

Developers like using the product

Do developers log in just to do development and test their work? Or do they love logging in just to navigate around and get a little dopamine boost from it?

A happy developer is an even better predictor of customer satisfaction than a happy product owner.

These criteria, of course, describe a semi-perfect world of unicorns and rainbows. They depict an ideal, desired but rarely achieved. Corporate decisions are often made very differently. But that reality should not preclude us, technical people and especially architects, from having a good set of criteria and rules to guide us in formulating a plan to get to prod and to make our product better after the release.

Paradigm mixing

I like my hot beverages at instantly sip-able temperature – above warm, below hot. Yet I do not want to add calories to each hot drink I take, so milk is not an option. When I make my own coffee, I know what to do. But when I work in an office that has a coffee shop but no DIY coffee station, I am forced to develop a request interface to convey my need to a barista. I began by asking for ice cubes in my coffee. 8 ice cubes in a large cup. 8 ice cubes. That got more eye rolls and comments than I expected. The same baristas that are happy to make you an iced tea or coffee roll their eyes when you ask for a cup of hot coffee with 8 ice cubes in it. Why?

I thought – ok, 8 is a neurotic sounding number, perhaps rounding it up to 10 would appease the baristas. It did, marginally. 10 was better than 8 but I was still getting the annoyed facial expressions.

I was not giving up. 8 was too weird, 10 was too specific-sounding, – I reasoned, – so maybe asking for “a few” ice cubes – taking a precise number out of the request – would help matters. It did not. The baristas were less annoyed, but I was still not getting a desired result. A few apparently means 2-4 and several means 4-6. My coffee was never going to be just right.

There is no right way to ask for a hot coffee with some ice cubes and get exactly what you want – without agreeing to be viewed as a “strange guy”. It just seemed like a manifestation of a general human trait rather than bad customer service.

We like things to be familiar. We like patterns. We feel safe when everything around us makes sense. There are concepts we grow accustomed to, for better or worse, and any deviation from them puts us at unease. To apply these generalities to my coffee problem, I was trying to mix two paradigms that usually live in their separate worlds. There is Hot Coffee (would you like some room for cream? – they ask). There is Iced Coffee (fill the cup with ice, that’s right, all the way to the brim, then pour hot coffee). Hot coffee with 10 ice cubes is a mutant. It is a paradigm mix and anybody asking for such a thing must be an alien.

It has been a common experience for me to get a “nobody has ever asked for this before” remark.

Paradigm mixing belongs in the world of heroes. It is the world inhabited by the paralympian legless runner, the 100-year old marathoner, trucker-turn-Elvis, one-handed violinist. It belongs in the news and in the movies. But taken to its trivial level, paradigm mix scares us. That is one of the main reasons most people find it so difficult to come up with an original idea. An original idea is very often about taking two things that exist separately and making them into one and very often it is about enablement.

Think about it. There have always been people who lost limbs or eyesight. The history of sports competitions go back millennia. Yet somehow, until recently, nobody thought of putting the two separately existing things together and making a truly remarkable human and business venture of it.

Self-publishing, the phenomenal success of YouTube, social networks becoming huge financial successes are all testimony to the idea that people are as diverse and creative as the environment allows them to be. Give people tools to express themselves and the society will benefit.

How does this relate to my perfect cup of coffee problem? Simple. The modern coffee shop is stuck in a rigid model of yesteryears and is not willing to become creative. Give people tools, put a little fun into getting coffee and the business will be booming, I am sure of it.