Few products have affected the modern world as much as the iPhone. It’s one of the best-selling devices of all time, and helped usher in the mobile-computing era that now has become the de facto way that billions of people around the world communicate every day.
The iPhone was announced in early 2007, but a group of engineers, squirreled away in a secret facility within Apple’s campus, had been working on the device for years, sworn to secrecy and reporting directly to then-CEO Steve Jobs.
One of the principal software engineers on the team, Ken Kocienda, wrote a book, released last week, on his time working on what he calls the “golden age” of Apple design. He’s credited with designing the first software keyboard for a smartphone, and helped transform the smartphone from clunky, small-screened devices like the BlackBerry, into the sheets of glass most of us carry these days.
Kocienda worked on the product that turned Apple from an innovative computer company into the one of the most profitable companies in history, one that generates more than $100 billion in revenue from the iPhone each year.
Quartz sat down with Kocienda, right before Apple unveils its latest iPhone, to talk about his book, and what it was like working on Apple’s most important product to date.
This interview has been lightly edited for clarity.
Quartz: What was it like working on the early iPhone team? How much of what you were doing was turning theoreticals into real products?
Kocienda: We were asked by Steve [Jobs] to make this touchscreen operating system. So we did what Apple has done throughout its history: some new hardware technology that becomes available that opens up some new capabilities, and so people like me and the team that I worked with were asked to come and provide the software component for that, to take multi-touch [gestures] and build a system that people could use for general-purpose computing, to carry around in your pocket.
The way that we approached this goal was again in a very Apple style, in that whenever we had an idea for a new piece of software or new feature, we made demos, and even if those first versions weren’t any good, we figured out right away how to make that first version better—how to take the weak parts and throw them away, recognize the strong parts and build on those—and then make the next demo, and keep going and going. And the great thing about the culture at the time was the very small, very cohesive teams, and with a product like the iPhone, Steve was paying attention.
This had the effect that, when I was working with a colleague in the way that I just described, making these demos and trying to figure out how to improve them at a very early stage, there was also part of this process where we showed work to managers, and executives, and to Steve. We would get this these decisive responses in this feedback and this criticism that helped us to then make the next version and the next iteration, to just keep going and going—it was like a Darwinian process.
We evolved the products—even though they maybe didn’t start out so well, they ended up being, I think, pretty great.
How close would Jobs manage things—would it be down to the the minutiae or was it broader strokes?
He didn’t manage people like me. I was an individual contributor; a programmer assigned to write code, to work with the designers to come up with these demos and prototypes, and eventually the product. He didn’t micromanage; he set assignments. He said to me, “We need a software keyboard for this product, because there’s never going to be a hardware keyboard like a Blackberry, so let’s go and do it.”
And then it was my responsibility to figure out how to make that happen. He saw work on a regular basis, usually every week, so he was always in touch with what the latest work was, and he would respond. I like to think of it that he positioned himself as “customer number one.” If you did a piece of work and you showed it to him, he would respond to it like a customer in the store as if it was the real product. Even when we were at an early stage, he’d tell you what he thought. And then we were responsible for going and addressing that feedback, trying to figure out how to diminish or even eliminate the weak parts and build on the strong points.
Did you ever have the thought that you wouldn’t actually be able to deliver on what was asked?
Sure! You never know. You never really know how things are going to turn out.
We were committed to get the work done, but if you’re really pursuing a creative process, you don’t really know what the answers are going to be until you have them. So yeah, there was always worry, there was always concern, that we weren’t going to make it, that maybe we wouldn’t be able to come up with the ideas that were necessary.
Of course hanging over me, in particular, doing the keyboard work was the shadow of the Newton. Apple made a product in its history that failed because the text entry wasn’t good enough. I was worried about that and I just worried that we wouldn’t come up with good enough ideas to make the whole product to come together up to Apple quality.
Other than the screen size of phones, the basic structure of your keyboard hasn’t changed that much over the years. But have you thought about what competitors have done since then, like swiping keyboards, and wish you’d had something like that at the beginning?
I do think the way Apple is continuing is the right way. I think that swipe keyboards are very interesting, but there’s a learning curve. For gadget lovers, it’s great, if you have the interest in investing your time in how your device works.
Most people are not focused on the device that they carry around. They just want the thing to work. They just want to text her friends. They just want to surf the web or look at some photos. They don’t want to use the phone as a technology artifact. It’s just a thing that should be getting out of the way.
I have no problem with other technology companies, programmers, and designers thinking of other ideas for keyboards. Swipe is a great example, but I think that for most people the QWERTY keyboard—where you tap the ‘A’ key and you get an ‘A’ that people are familiar with that doesn’t require much explanation—still is the best solution.
I type fast enough, but I’m always amazed by people who can type reams on their phones—it’s a feat I’ll never achieve.
The funny thing is, of course, I invented auto correction, but I’m a lousy touchscreen typist. From very early on, even when we still had our tethered prototypes, when you needed a whole Mac and an open computer board on your desk to make the thing work, there were people who could type with two thumbs. It was like, “Holy cow!” And there was one guy in particular who took to it faster than anybody else, and we always had him come in for performance tests—is the software fast enough, can it keep up with the speed that people will be able to type?
One of the other interesting aspects of working in that small environment is that somebody like that one typist represented potentially millions of people out in the world. We had these very small development teams with just a couple dozen people. If you had one person who seemed way, way out on the end of the distribution, it turns out that, yeah, once you multiply that by the number of people in the world, there are a lot of fast typists out there.
And a lot of slow typists.
A lot of slow typists, too. And so, of course, the challenge with Apple products, like text entry, is to give people a technology that will work for both the people who can just burn through and write a novel, or if they’re a little bit slower, more deliberate, that is fine, too.
So was it always the plan for us to type on iPhones with two thumbs?
No, except to the extent that this was kind of a BlackBerry style, but we didn’t confine our ideas to that. It was years later that I actually took to two-thumb typing. Naturally there were people who approached it that way from the very start, but I actually was a very slow typist—I just use one index finger, hold the device in my left hand and go at it with one finger. I’ve upped my game since then.
What was it like building autocorrect for the phone? Was it just a lot of trial-and-error, sitting there trying to get errors to get the right replacement words?
The key idea for auto correction was correcting your near-misses on keys.
So, “I” and “O” are right next to each other on the keyboard, and the trick was to figure out which one of those you meant. It’s this whole idea of giving people what they meant rather than what they did. And with a hardware keyboard, you can offload that responsibility onto the user. If you make a typo, you can feel it with your fingers, whether that be on a desktop keyboard, or a laptop, or a BlackBerry. That went away when the keyboard became a sheet of glass, and so the software had to step in to figure out well, OK, given the pattern of keys that you made with your taps one after another, what do you mean? The software challenge was to make that determination better and better so they could understand, and thankfully this was possible, after a lot of work and and trying to understand what people’s expectations were for the kinds of errors they would make.
And presumably it could get better over time as well. The more users you have, the more feedback you have.
Right. Even from the very beginning there was what I called a “dynamic dictionary” of what worked in concert with the static dictionary. The static dictionary was a list of English words that didn’t change until we updated the software on the device, and the dynamic dictionary was looking at the words that you typed. So if you had a bit of jargon in whatever field that you worked in—maybe an inside joke with your friend, a funny word—the keyboard would learn it. The dynamic dictionary would learn that’s a word for you.
Was there a debate about whether to include swear words or not?
There was a conversation—a long one. Where this came down was that even though people maybe found it frustrating when they didn’t mean to type “duck,” but that’s what they got. The design decision that we made was really skewed toward the opposite.
What if you meant to type in “duck,” but you typed the other word? We didn’t want to offer helpful auto-corrections to have you come across being a foul-mouthed at grandson when you’re texting your grandma. That would be worse.
Was there ever any thought back then about the awkwardness for users of typing on glass for the first time? Was there a concern people would just want to stick to physical keyboards?
Yes, we were worried. We referred to the keyboard sometimes as a science project. We didn’t know. It was kind of the software equivalent of gurgling test tubes. What would happen when we put these two ideas together: is it going to be some great new compound or is it going to explode?
The thing is, though, that the keyboard had to be in software for the device to achieve the potential that I think Steve had for it. With the keyboard in software, it could get out of the way when you weren’t typing, and it just opened up the device so that the whole screen would be taken up by an app, and as it turned out, I think the iPhone was successful from day one, but it was when the App Store became available that product really took off. The keyboard was just a software feature that was a piece of that puzzle.
Do you think that the process of designing the integral pieces of software like that has changed over the years at Apple, with Tim Cook taking over as CEO from Jobs?
Things have changed because Steve was such a unique personality. He was involved in looking at the software all the time. Other people have stepped into that role, and Apple’s a trillion-dollar company now, so I mean I think they’re doing OK.
And there’s always been this strong cultural thread running through Apple of evaluating the work as we go, trying to look at new technology from the perspective of people out in the world, trying to make the best products possible. So that hasn’t changed. Even though some of the people like Steve regrettably have.
Do you think that what you achieved at the time would be possible today? Is Apple still set up to create products that could be as revolutionary as the iPhone was?
The iPhone, to me, was this opportunity that doesn’t come along so often. There was this moment where the form factor shift became possible—where computer hardware became powerful enough, CPUs, GPUs, and batteries were good enough, multi-touch screen technology was there—so you had this whole hardware foundation in the raw. And then [the cellular] network was getting good enough so you can have data with you all the time. The contribution that we made was then taking that hardware and networking and applying this software, and defining what this new pocket computer could be like. This opportunity was mostly not of Apple’s making, but Apple recognized that there were these pieces that could be combined in this new way to make a general-purpose device: a computer.
That potential presents itself, for that form-factor shift, only so often in the history of computers.
Does hardware development tend to lead software development at Apple?
Yeah. Working in new product development, there’s always a lead time. I worked on the iPhone for a year and a half before it was announced. And there were some people working on touchscreen technology a little bit before that, but pretty much we were presented with this blank canvas of hardware capabilities.
So hardware does in many respects lead now in some cases. Maybe even today at some lab Apple’s squirreled away totally secret, there’s some software person sitting with a hardware person saying, “Hey, maybe a couple of years from now, maybe we could deliver this—what software might you be able to write for that?”
I think that is a kind of an approach, where you try to combine both hardware and software to create these new experiences.
How does a commitment like that work at Apple? Is it… a hardware team in the lab realizes what they have, and they bring it to their managers, they bring it to executives, and then they commit resources to it?
Yes. That’s pretty much how it works.
Of course Apple is very compartmentalized. My plate was full with software issues and features and ideas. But in the case of a product like the iPhone, I was issued this handheld unit that we call the “Wallaby.” It was the codename for the prototype touchscreen hardware and it was a display that responded to touch, but you needed this tether coming out of it that connected to this computer board. I had a bare computer board sitting on my desk for basically a year and a half that was then connected with other cabling to an old blue-and-white Mac tower desktop computer, which provided the computing power to drive the display.
All of that gear together made it possible to start experimenting with the software experiences, that I could write some code, run it on this desktop Mac, and and have it show on the handheld device as an external screen.
So it’s not like anyone could’ve easily walked out with one of those early prototypes then…
Ha, no, we didn’t have this problem. In later years, there was a bit of a joke that somebody pinned up near the exit at one of the engineering hallways: We’re in the engineering lockdown and somebody took this big sheet of paper and wrote: “Wallet? Keys? Prototype?”
For a long time I had these hardware prototypes that it would have been the end of my Apple career if I even talked about, never mind walked out of the lockdown area with them. But in these early stages, where you’ve got cabling everywhere, it would’ve been a bit conspicuous to have been some guy with a trench coat and some Quasimodo-like hump on your shoulder in order to make off with it.
But how tough was it, working on a team like that, where you couldn’t talk to anyone about it?
We couldn’t tell friends and family—you just couldn’t. But the thing is that, inside of the secret labs, the lockdown areas, everyone you saw in there was disclosed and so we had a wonderful working environment in there. There was a real sense of small-team cohesion; we felt like we were on a mission together. In there, it was a very open environment where we talked about all the problems that were on our plate at the time.
We even thought about funny little ways of talking about it even in the cafe. You’d say, ”Well, we’ve got a demo later today for ‘the thing,'” because even though we had a code word for the project (purple), you couldn’t say the code word outside of the lockdown area. It’s kind of funny—why did we have a code word if it kind of defeats the point? But the culture was what it was.
But inside the lockdown area, I always felt like if I had an idea and I wanted some feedback and wanted to get a response to it, there was always somebody you could go to their office and show them the latest demo. (Or in those early days call them into your office because you just couldn’t pick the thing up and move around with it so easily.)
Was there anyone you particularly enjoyed working with?
In some ways it’s unfair because there were so many wonderful people on the team, you know people like Bas Ording, who was one of the human interface designers. He’s the person that I worked with most closely on the software keyboard. He’s the one that designed the final look and feel of it, and he’s the one that decided the font sizes.
Bas and I together decided where we put the punctuation—there’s no room for it on that first keyboard—well, you type this little “period, question-mark, one, two, three” key and we decided that was going to be the characters on that key to let you know that there’s punctuation and numbers on this other key plane. We worked together on that the look and feel aspect of it and it was a real pleasure to work with him.
How did you decide what keys to include beyond the QWERTY keys?
There was some analysis, but you know in some ways, the very first version of the iPhone was easy, in that it was only for sale in the United States. Very soon after that I started working on additional languages—French, German, and Japanese were early on the list of languages to add—and that’s where I came up with the notion of “press and hold” on a key to show the additional characters, like accented characters, or press and holding on the dollar sign to get the euro sign. The signs were tied to the markets that we were going to be releasing the phone in early on. But other than that, we used our taste.
Looking back after a decade to where the iPhone is now and versus what it was when you first started, is there anything you wish you’d known then?
One of the things that has become surprising to me is how much the phone has become this device that people look at so much throughout their day, even to the point where some people think it’s a problem. “I’m addicted to my phone” is not an idea that I personally ever considered. I think that it’s part of a cultural evolution that I think we need to maybe get better at figuring out the right social norms. I think we pretty much know now that if I were to take out my phone right now in the midst of our conversation and look at it, that would be inappropriate.
I think that if we had been able to see how that would go, maybe we would have been able to think of software features. We’re now beginning to see that with Screen Time on iOS and similar features on Android devices that are letting you know how much you’re using your device.
I think that knowledge will help people because, you know, my goal was to make the device useful and meaningful to people. I just didn’t see that the maybe there would be a flip side to that.
Do you still use an iPhone?
Of course. I do. I’ve got an iPhone X, I love it.