Bots and the (work) place

By Susan U. Stucky
Issue #10 August 2018

Tags: conversation • work • chatbots • augmentation

Begin with this brief conversation about the article below – between Work&Place Managing Editor Jim Ware and author Susan Stucky:

The Article:

Getting work done takes a lot of interaction – coordination, cooperation, collaboration – not just between and among people but between and among people and aspects of their built environment. That interaction can be between a person and “the cloud” mining bitcoins, or people standing around whiteboards trying to figure something out. It can be talking over a cup of coffee or tea in the kitchen or holing up to get some heads-down work done.

As we are learning in customer service interactions, it can also be a conversation with a chatboti – a bot being a just a chunk of software that, when it is invoked, people imbue with agency. We respond to the automated voice that asks us for our telephone number for identification purposes and additional software continues to do the identification.

It is worth pointing out that the built environment includes digital technology, though sometimes we seem to forget that and relegate it to the realm of the virtual as if it didn’t exist. But digital technology does exist. Recently, news headlines about the staggering amounts of electricity used by servers in bitcoin mining have served to remind us of the physicality of the digital.ⁱⁱ As with the introduction of any technology, digital technology has brought with it new forms of social interaction, the rise of more conversational modes and, hence, new ways of getting work done.

There is a range of digital technologies finding their way  into the places people go to get work done. Ambient technology that suddenly becomes relevant, for example the thermostat on the wall.

What happens when the whole room starts talking to the thermostat (“Hey, listen to us, turn the temperature down!”)? What happens when the thermostat starts talking back (“Hey, I already did!”)? Then there are those data centers in Iceland and server farms powered by hydroelectric generators on the Columbia River that run between Oregon and Washington state.

The “place” in which that technology resides has less to do with getting work done (unless, for example, you are in charge of configuring the server farm or involved in maintaining the cabling that runs to it). Rather, human conversation with digital entities has something of the here and now to it. It has presence.

An IBM TV ad shows a conversation with IBM’s Watson as face-to-face with a finance executive.iii While this advertisement signals several different things to the viewer, one is that Watson is present right there, right then, together with the executive, and they are in conversation. (The executive’s work? In this case, part of making a decision about a vendor, one presumes.)

People can learn a lot through conversation. Conversation is often where decisions are made. It is a place where knowledge is constructed, where things are understood. Conversation, that very special form of human interaction, has always been key to getting work done. It is surprising that we continue to assume that certain kinds of work do not involve conversation, such as writing a computer program or writing a book. Yes, there is heads-down work involved, sometimes quite a lot of it, but neither the program nor the book will get written. The work will not get done without conversation with other people.

In point of fact, the design of the (work) place has always been interwoven with the “sociality” of getting work done: meeting spaces, white boards, hallways for mingling and serendipitous encounters. Places for teams to meet provide persistent context available as shareable context when the team occupies it for weeks or months. Instant messaging, a digital technology instrumental in getting work done, facilitates conversational interaction as well. AI (Artificial Intelligence) now makes it possible for people to work together with digital entities. The participation of digital entities in people getting work done is here, whether we or the places we choose to get work done in are prepared for them or not.

Conversation is a lovely thing when it goes well, but it is full of missteps and misunderstandings even among humans. We seek clarification, we make repairs. Why should we not expect the same to happen with our digital conversation partners?

Conversation is a very local phenomenon. It only works well when there is shared, or some might say, sharable context. It can take a while to realize that the help desk you have reached is being serviced not by the company you bought your computer from, but by some other company, or by a consultant hired by a third company, and to discover that no one in the service network has imagined the problem you are having. What are the chances of getting the problem solved then?

Problems of reference abound too. It can take a while to agree exactly which particular laptop, or is it a tablet, is being talked about. Good luck if you should give the wrong serial number because the tag is too faint and scuffed up to read any longer. Or if you rely on memory as to which operating system is actually running. An endless loop can ensue. We should expect to be able to clarify things with our digital conversation partners. We should expect the digital entities to act in accordance with human expectations.

This article makes two points:

1. It behooves designers of the built environment to make use of what is known and understood about conversational interaction in the context of getting work done. At least to know that there are whole domains of expertise – computer dialogue systems and conversation analysis to name but two — that are relevant to the design of the built environment and to seek them out.
2. It will be essential for designers to be clear both about the context of the digital entity they are conversing with and the content of those conversations – what the entities are talking about. This is a big ask since traditionally it has not been a requirement of system building. Apps as presently designed, for instance, do not have requirements about the contexts they will be used in. Yet their successful use relies on a particular, specific context.

Though people, both professionals (media included) and the public, talk about (and hype or disparage) AI in general, these new kinds of interaction up the ante on design. Designers will need to remember that digital entities, just like humans, operate in the particular.

Conversation and getting work done

Suddenly, it seems, we are in a world of bots. Chatbots, web robots or WWWbots, chatterbots, IM bots, persona-bots, and then there is Alicebot, the Artificial Linguistic Internet Computer Entity. “She” has been around since the 90’s helping make chatterbot interactions better and better. There are also “bad” bots, such as those in a bot net that infects your computer. (While the difference between the good bots and the bad bots is, in some way, in the eye of the beholder, malware is not the subject of this article, as pressing as that phenomenon is).

Before we called bits of software that we interacted with ‘bots’ they were already evident. We know them as triggering replies to messages when someone is out of office, asking for information. Recommendation engines, as they are known, recommend books that others bought when they bought the one you did. (One colleague jokingly complained when a recommendation engine was first used by Amazon, that he had to convince a friend that they didn’t need to buy all of the recommended books). While they are easier to ignore than someone in a human conversation, they are not that easy to ignore altogether, playing as they do, with social norms generally associated with human conversation.

Now, the likes of chatbots, chatterbots, and Instant Messaging (IM) have made bots both more present and more conversational. They “demand” a response. We are used to giving responses, and so we do, even as we wish the pop-up window wouldn’t pop-up. Remember “Clippy” the Microsoft Word assistant of the late 90’s and early 2000’s? Perhaps it wasn’t conversational enough.

To start with, conversation plays a significant role in getting work done, bots or no bots. It’s a fact that seems to get overlooked in our insistence on only seeing and measuring what individual people do at work, what skills they each have. Time on task seldom takes into account actual human conversation. In fact, talk with others is sometimes considered (by both management and business process models) as time not on task, even though conversation is essential to getting work done efficiently and effectively.

Computer programming, for example, is often thought of as individual work done by an individual person that can be done any place, anytime. But that perspective cuts out a lot of what a programmer actually does at work. A recent piece in the NYT (New York Times) illustrates how conversation is part of getting work done. The story, recounted “as told” by Samara Trilling to Kevin Roose (a business editor at the NYT) who works at Sidewalk Labs, which makes and uses technology to improve “the way cities work.

“Most days, I wake up in the morning, take the subway to work, grab coffee or tea in our work kitchen and start having a conversation with a colleague about some problem we’re working on…I sit at a table with 12 other people…writing things called methods or functions that solve really small problems…we do a lot of pair programming – that’s when you have one person typing code and one person looking over their shoulder. It means that if I have a question, I can ask my co-worker immediately.

Engineers need long, unstructured blocks of time to work without interruption…[Note that being able to ask a question of a co-worker apparently does not count as an interruption.] Code reviews are another big part of my day. After I’ve written a piece of code that I’m happy with, I submit something called a “pull request” and all my colleagues can see it…We have “retro” meetings very week…Two or three times a week we have ‘stand-up’ meetings…another big part of my day is communicating with non-engineers. You have to decide what to build with other people. When you have the knowledge of how computers work, one important job is explaining that to other people.”

That hardly looks like working alone, notwithstanding heads-down time. Ms. Trilling reports talking with other people, sometimes virtually, sometimes face-to-face. If you are looking at how work gets done “in the wild” then the sociality of work is inescapable.6 And conversation, whether in meetings, the kitchen, at the twelve-person table, even while programming, is a big part of it. And now we talk with and to digital entities of all kinds.

What impact does the sociality of getting work done imply about our physical work spaces? And when that sociality includes digital entities, what does that imply about the places we work?

 

Time on task seldom takes into account actual human conversation. In fact, talk with others is sometimes considered (by both management and business process models) as time not on task, even though conversation is essential to getting work done efficiently and effectively.

We have always wanted to talk to our digital entities

People have wanted to figure out how to talk to digital computers from the very beginning. There is the fabled ELIZA, a computer program that was designed to emulate a Rogerian psychotherapist back in the mid-60’s. Now we have Alexis, Cortana and Siri, so-called smart assistants, sometimes called AI’s (artificial intelligences) and now “smart” digital home assistants, Google Home and Amazon Echo.

At the present time two conversational models predominate in this space: rule-based bots that follow scripts and AI-based bots that aim for more flexible, more natural, interaction. Both models now use either text or voice. Conversational interfaces are getting better; who hasn’t wondered whether they were texting with a chatbot or a real person at one time or another

To date, mainstream uses of conversation with bots have been mostly about gathering information or answering questions, (e.g., surveys of one type or another, or customer support) both things people do when they are in conversation with each other.

There is another use for conversation that is just now making it into the mainstream, and that is conversation in the context of getting work done. Think about medical diagnosis being done through Q&A by IBM’s Watson Care Manager with medical professionals.v Or DF 2020’s Chatbot Author,vi which is used to construct real-time conversational interaction among medical personnel and chatbot-enabled procedures.

Conversation, that very special form of human interaction, has always been key to getting work done.

We have always wanted our digital technology to make getting work done easier

Just as people have always wanted to talk to digital computers, people have always wanted digital technology to make getting work done easier. Automating processes, whether in manufacturing^vii or finance has moved along pretty steadily over the past 60 years.

In manufacturing, digital control of machines helped increase accuracy and decreased injuries of humans while increasing productivity. Automation in finance has likewise increased accuracy and decreased the drudgery of data entry while increasing productivity. At the very same time digital automation got off the ground, a second point of view emerged: viewing digital technology as assisting or augmenting human work.^viii

These two views, automating work, as exemplified by digital spreadsheets and mortgage qualification and augmenting work as exemplified by the computer mouse and automated braking systems, have existed uneasily side-by-side all these years,ix although recently, with the rising awareness of job loss, the concept of augmenting or assisting human work has again received attention.

During that same 60 years, another shift has been taking place – the appreciation of knowledge work as a distinct kind of work, and its rise as a dominant form of work. Peter Drucker is credited with creating the term ‘knowledge worker’ in 1959, noting the difference between working with physical materials and working with information.^x

To the extent that we view all things digital as information-based, it would seem that all work is headed toward being knowledge work, at least in part. This isn’t the place to pull in all the philosophical debates about what knowledge is and how it is acquired or used. However, it is clear that the sociality of knowledge is exposed through interaction, and conversational interaction in particular.

Conversation turns out to be an exquisitely structured human activity. A recent book by N.J. Enfield, How We Talk: The Inner Workings of Conversation, makes this abundantly clear.^xi

Take the timing of taking turns in who is speaking. English speakers can initiate a turn, can respond to someone else, in a quarter of a second. A half second seems long; it makes the respondent seem to be hesitating. Danish speakers typically respond in a half a second, not a quarter second, resulting in the perception of Danes speaking Danish by English speakers as slower. That’s only 250 milliseconds longer, the point being that 250 milliseconds, just a quarter second, is enough time to engender that reaction in English speakers. The timing of digital entities’ responses may well be something that needs to be paid attention to, if it isn’t already.

It is important to note that the findings that Enfield reports are about naturally occurring language, not language “in the lab.” For much of the last 60 years the study of naturally occurring language was deemed off limits by Chomskian followers. Noam Chomsky, the 20th-century linguistics czar in the United States, declared early on that actual language use wasn’t going to get us anywhere in understanding the cognitive basis of language.

However, not everyone, socio-linguists in particular, stayed away from naturally occurring language. They contributed to the field of Discourse Analysis, drawing insight as well from the Philosophy of Language. Methods of Conversation Analysis, originating in ethnomethodology in Sociology, have greatly informed how to study human conversation.

Of special relevance to the topic of this article is the work on discourse in Artificial Intelligence, in particular the more recent work on collaboration with digital entities (e.g., robots). Each of these fields has something to offer the design of conversational interaction with digital entities in the workplace. Technical developments in AI and Computer Science have made analysis of large naturally occurring data sets much more feasible and can now be used to understand the interaction of human and digital entities when they work together to get work done.

For a long time, we have analyzed how work gets done only by looking at what individuals do. This focus misses the sociality of getting work done. For instance, a study “in the wild” around the quality function in a manufacturing company revealed in one facility that it was not the six or seven people technically responsible, but rather about twenty-four all told.

In another investigation, (in the context of the failure on the ground of an automated costing application for outsourcing), it was discovered that the people doing costing had kept on using spreadsheets (and not the new automated tool that did not support spreadsheets). The spreadsheets had been jointly created over time among the costers themselves (Note “jointly constructed”). One individual had fifty-seven (!) spreadsheets open at a time, doing “what ifs” for the customer. That’s not so many, someone once responded. He had found seventy!

Work Practice Analysis, another approach to figuring out how work is getting done “in the wild” reveals the sociality of getting work done: more people, and more tools and techniques, are involved than anyone (people doing the work, management or even management consultants) is aware of.

That’s partly because, as the philosopher Michael Polanyi is often quoted in explaining tacit knowledge, “We know more than we can tell.” And, the author of this paper would assert, because the sociality of getting work done is simply not acknowledged by organizations. It is not even seen, much less acknowledged.

Conversation doesn’t appear in workflow automation, business processes, or task descriptions. When road workers are standing in a group talking by the side of the road, an immediate reaction is often along the lines of “Why aren’t they working?” When was the last time you saw “ability to have and lead productive conversation” as a skill in a job description? To be sure, soft skills are becoming recognized as important.

Yet conversations have been key to getting work done all along.

Yet, even though there are methods for making the sociality of work visible, too often technology and space designers do not or, more likely, cannot, avail themselves of these methods of inquiry. It is said that they take too long and cost too much. Now that technology is going “wild” that may be even less tenable than it was before.

What about conversing with digital entities to get work done and (work) place?

At first glance, it may seem there is nothing to think about. As long as the digital entity or entities you are conversing with are there with you, or immediately available “virtually” and as long as there is a safe, secure place for them somewhere, what is the problem? The thing is, it isn’t the place where the digital technology is that matters, it is the context of its use.

The financial executive in that IBM Watson ad isn’t sitting in an airline club. And that screen Watson is visible on didn’t walk there by itself. Suppose the executive was in a hurry and had a laptop on in his hotel room and just clicked on a link to watch while he shaved. Probably not as productive a conversation, one guesses. This isn’t any different from an ordinary conversation where both parties are present

Chatbots have some of the qualities of human conversation, but quickly become useless as the context of use isn’t shared. Using Google Assistant to talk to a Nest Learning Thermostat is possible now, but its effectiveness is dependent on the context of use – the particular speaker talking with the Assistant and the Nest sensors being in a certain particular location. Co-constructing the context of use through conversation will be key to human participation with digital entities. What co-location does is help establish the shared context of use. Are we in the hallway or in the boardroom?

The answer to the question posed at the beginning is framed here as a hypothesis: to the extent that the context of use for conversations between and among humans and digital entities is shared, or at least sharable, the greater the likelihood of productive conversation.

Conclusion

Is the emergence of conversation with digital entities the inexorable playing out of the trajectory we have been on — increasing automation and assisting humans — or is something new happening? The answer is Yes. The entry of digital entities into the workforce is something new. It’s time to open the aperture, to move beyond individual behavior and individual performance to look at the inherent sociality of creating knowledge and getting work done.

It’s time to stop thinking that human behavior can be fully understood without understanding the sociality of humankind, the various forms of social interaction and configuration that constitute society. It’s time to address what it will mean to work with digital entities that actively participate in getting work done. We should take a cue from how we humans understand each other: co-constructing sharable context and sharable content as we go along.

Finally, as conversational interaction with digital entities moves out of the lab, and out of university research, it is foolish not to take advantage of the rich understanding of human conversational interaction. Without it, conversation with digital entities will be far less productive, even wasteful of human productivity

Susan U. Stucky

Work and Learning are at the heart of Susan Stucky’s professional career. Since retiring from ten years with IBM Research first as a consultant and then employee, she has continued her focus on work in digital transformation and the design of work marketplaces. She led pioneering work on space design for knowledge work and informal learning on the job. Her insistence on how people work and learn is based on how these activities actually unfold in the real world. That approach, she claims, provides a much better foundation for change, whether it is in the context of the current push for digital transformation or in addressing the challenges and opportunities of the changing nature of work and learning.

e          [email protected]

l           https://www.linkedin.com/in/susanstucky

 

References

i   A bot is a chunk of software that people regard as an agent. Some bots do our bidding; they act on our behalf, as in a computer game. Other bots are automated (by whom is sometimes not known), as with automated tweets.

Ii   https://www.nytimes.com/2018/01/21/technology/bitcoin-mining-energy-consumption.html retrieved 18 Mar 2018

iii https://www.ispot.tv/ad/A6ZY/ibm-watson-ibm-watson-dbs-bank-on-cognitive-finance retrieved 15 mar 2018.

iv    Roose, K. “What Does a Computer Programmer Do All Day?” New York Times, 25 Feb 2018.

v   https://www.ibm.com/watson/health/value-based-care/watson-care-manager/ (Accessed 5 Mar 2018).

vi   http://www.df2020.com/use-cases (Accessed 6 March 2018).

vii  Hutchins, E. Cognition in the Wild. MIT Press, 1995.

viii Nye, D.E. America’s Assembly Line, MIT Press, Cambridge, MA: MIT Press (2013).

ix  Engelbart, D. C. “Toward augmenting the human intellect and boosting our collective IQ” (PDF). Communications of the ACM. 38 (8): 30. (1995).

x  Drucker, P. F. The Landmarks of Tomorrow. New York: Harper and Row, 1959.

xi  Enfield, N.J. How we Talk: The Inner Workings of Conversation. New York: Basic Books, 2017.