This is a transcript from the AI and the Future of Work podcast episode featuring Chandra Khatri, Chief Scientist and Head of AI at Got It AI, discusses the future of NLP for better customer experiences with bots

Dan Turchin (00:19):
Good morning, good afternoon, or good evening, depending on where you’re listening. Welcome back to AI in the future of work. Thanks again for making this one of the most downloaded podcasts about the future of work. If you enjoy what we do, please like comment rate and share in your favorite podcast app, and we’ll keep sharing great conversations. I’m your host, Dan Turchin advisor at insight finder, the system of intelligence for it, operations and CEO of people reign the AI platform for it and HR employee service. Now the most expensive and inefficient way to engage customers is by having them speak with a live service agent who frequently places them on hold creates a ticket and then promises to follow up by contrast, the easiest way to engage them is using existing sources of knowledge to find answers to their questions automatically using natural language. We’ve always known that’s the best experience, but until recently we just haven’t had the combination of NLP technology or machine learning infrastructure to automate the entire process.

Dan Turchin (01:25):
Today’s guest is developing that technology that’s required to make customer experiences with bots, feel more like conversations with the best live agents. The insight behind companies like got it. AI is that the best answer to every customer question already lives in existing knowledge articles, or maybe historical tickets finding it can be tricky, but there should be a reliable way to turn data into answers in real time using natural language. Chandra Kari is the chief scientist and head of AI at guided AI, which develops tools for customer experience management. Chandra’s at the forefront of NLP research. Having spent time in senior AI engineering roles at Uber and Amazon after having received his masters in machine learning from Georgia tech and his undergrad from Bitz Palani in 2012, without further ado, Chandra, welcome to AI in the future of work, let’s get started by having you share maybe a little bit more about your background and Hey, you got excited about this space.

Chandra Khatri (02:29):
Thanks Dan, for the introduction and also for the invitation. So about myself, you already gave a good background on what what I’ve been doing, but to give, to talk about my journey, I have been in this AI space for over decade now, and particularly in the area of conversational AI and LP for almost yeah, almost I would say, close to etiquette. So it, the journey began when I was a Georgia tech and over there, I was building some of the applications in, and then there was a rise of, or sudden rise of conversational AI technology, such as Alexa in 2014 15. It was around that time. I saw that, you know, this area of NLP and conversational AI is becoming big and consumer space is just the area where things are starting up, but the impact is gonna be almost in every vertical it’ll be in it, it’ll be in healthcare, it’ll be in HR, it’ll be, there is no vertical, which will not be impacted by this.

Chandra Khatri (03:25):
And that led to the my, my journey so far. I built, I was the founding member of eBay’s conversational group and then jumped on to Alexa, AI, founding member of being the founding member of Alexa price. So that was a great, full experience for me, Alexa, I and Alexa price. We built a lot of good stuff there. And then after that I, along with some of the leading experts in conversational AI, we formed a conversational AI group at Uber AI where we built some of the conversational applications for drivers and rider at Uber. And which is very important because in Uber, you know, while drivers are riding their cars we, or any vehicle, we have to sort of empower them so that they’re safe. And also they can and some, some countries have you know legally they’re not required to interact with the phone or interact with the, let’s say right as while they’re driving, right?

Chandra Khatri (04:18):
So we wanted to make this application very easy for them. And that’s why we build a lot of these conversation, AI applications for them. And then it was around that time. I I met with Peter Lynn, who is the chairman of guard AI. We, he asked me a question, well, you know what if you want to build autonomous conversation, AI system, what is needed. So I told him, you know, it’s, it it’s in the near them. It was not possible, or it was not there because the way this entire ecosystem is built, but there’s a way. And luckily transformers technology happened to come and we, we chatted and chatted and I started advising. And finally I joined a few years ago and then we have built world’s first, fully autonomous conversation, AI system. But, but that, what means is that as you said, we can build a conversational experience out of any historical logs.

Chandra Khatri (05:06):
It could be email tickets, it could be chat, it could be voice transcripts or voice logs, or it could be even FAQ articles. What we do is we consume any kind of interactions, historical logs, or interaction or existing data such as articles, FAQs, we consume them, or our technology consumes them and automatically gives you a conversational experience, which typically takes weeks to months for enterprise and many, many months for actually large enterprise. We give that experience within few hours, few minutes, few hours to few days, max. So that is, that is our technology that we are aiming and we have built. And that is what we’re going after.

Dan Turchin (05:43):
Before. Got it. AI, you were at three of the most iconic AI cultures or early leaders in AIn eBay, Amazon and Uber. Tell us about the different ways that technology gets developed there and what you learned from the cultures there as you took over the role at got it. AI.

Chandra Khatri (06:03):
Oh, that’s a great question. So I would say I’ve been in different eras of AI. So eBay was one of those where the application around 20 13, 20 14, 20 12, that was the time of recommendation system. Most of the AI and machine learning was for search ranking, better search. You know, that was the era. So I’ve, I’ve been in those that time where that’s where the AI was. Then when I was at Amazon you know, Alexa, when we were founding main for building blocks of Alexa at that time the technology was slightly shifting towards more conversational and consumer oriented applications of NLP. So so therefore the product was different. The experience was different. The interface was voice. And as you know, since 20 17, 16 gig economic came happened to come into picture, you know, Uber Lyft, you know, in stock car, bunch of other of these, you know, gig economies are coming up.

Chandra Khatri (07:00):
So Uber was that experience where the applications moved from not only AI applications, but you see as a, through your devices, it was more towards the entire ecosystem and operations, right? So now AI is moving from AI, move to machine learning, moved from search ranking to conversational experience of products now to more operations as well. And the next wave, which I am right now at guard, AI is making almost even executives and large enterprise more powerful because these companies do not have any AI capabilities. The idea is to take AI from operations to the entire supply chain entire chain, right? And those application area could be it healthcare bunch of others, HR, as, as you mentioned, right? So these are the different verticals. We are going to empower them more and more. And the technology as you see, is moving from in 20 14, 13, before that the technology was centered only towards large companies, you know, eBay used to be big at that time or Google or Facebook, you know, so the technology was there only in the hands of those, right then 16, 17 open source happened to come like in the AI space.

Chandra Khatri (08:05):
And now it is, you know, bother, there is no code AI stuff, which is making everyone, which is empowering almost everyone. And soon there are more technologies coming up, such as what we build at got AI that will empower even CIOs or product officers or PMs, because then they who do not have as hands on in technology, they can, they will be able to build all these tools with underlying AI doing the work. So that’s why technology has evolved in terms of culture. I would say all of the companies have different cultures. In fact, before either I was at Oracle, Oracle was even it was before my grad school. So I’ve seen different wave and different way people have interacted the entire ecosystem. So I would say each company has given me a different perspective. For example, eBay gave me a perspective on how to optimize on rankings, how to optimize on the last few bids, you know, that, that, that the final layer of only 0.1% 0.2%, that kind of improvement Amazon kind of gave me a perspective of leadership principles as Amazon is known for its leadership principles, how things how to take a big idea and break it down, and then through how to sort of lead with your team in terms of how to live leadership principles that Amazon is especially also on their documentation, the way they do PR FAQs and stuff like that.

Chandra Khatri (09:26):
And then when I was at Uber, my my, the culture, which the culture where Uber at that time was how to be more and more equal or, or I would say how to make things, how to make this entire community of rider drivers, in fact, users and everyone, how to take everyone together. Now, again, I was moving more towards that kind of culture where we want to respect everyone, and a lot of things have happened. And then at guard eye the culture, again, it’s entrepreneurial. So I’m learning just everything, you know from business development to even recruitment a child it’s it’s like startup is its own own different journey where I’m almost playing different hats or in, or having different hats or playing different roles in a way.

Dan Turchin (10:11):
Now, all AI is really a data problem. And at eBay and Amazon and Uber, you had a big data problem by contrast, I’m gonna guess that it got at AI when you’re dealing with enterprise problems, you have a small data problem. Talk to us about the challenges when you have less data, how do you build reliable, scalable AI without the benefit of limitless amounts of data like you had at those big companies?

Chandra Khatri (10:38):
Great question. And in fact, timing wise, this question becomes more relevant because you see, as, as you pointed out before 20 17, 18 technology was centered with, or was with only large companies, which had data now with open source and tools like hugging face and bunch of other open source tools by flow, and a lot of open source data available you know, because of the web people people can train NLP models on the data, on the web Wikipedia, Reddit, et cetera, on image. There is so much data on the web now. So data has not become as big of a problem if you, if the applications are general applications, right. That as we tend to go from a specific application for enterprise data becomes a problem still. So how let’s say, if I want to have an autonomous system for, let’s say healthcare, how will I build it if I don’t know their vocabulary?

Chandra Khatri (11:35):
How will I know if I don’t have the access to the schema? So that’s where this new transfer learning technologies come into picture. So these days what is happening is we, as a researchers, we train our large models or on public data sets or using state of the technologies. And then what we do is something like transfer learning. We take that technology and we only fine tune, or just modify a little bit for the end user enterprise. There still is a gap, but but you know, we just want to make that minimal change. And I give, I give this example of, you know, often, you know, you, you like, for example, pizza, take a pizza. So Italians, or maybe even Greek some say, but they have cracked the problem of the, how to make the pizza. So, you know, the base, you know, what is, what goes into it?

Chandra Khatri (12:24):
There’s bread, there is cheese there’s sauce, but now people can take that and make variations of those, right? Those, right. So some people might like you know sausage on top. Some people might like the GS on top. So the base is that, that the base part is common. It’s available to everyone. That knowledge is available, that more, those models are available. Now, people just can put their toppings on the top, you know, whatever the toppings they like, or they prefer. So that is what is happening now. So data is still a problem, but it is for the final milestones or the last leg, or, you know, when we need to make these applications for those enterprise. And that also relies on heavy data or larger data, but it’s not in the orders of millions of samples, but these days it’s in the order of thousands, or maybe even hundreds, depending on the kind of problem,

Dan Turchin (13:08):
Almost all of us that have spent time around NLP technology in the enterprise have been frustrated because of the slow adoption. What do you think is holding back broader NLP adoption in the enterprise? Is it the technology? Is it cultural? Is it something else? And to the extent you may not even agree with that statement, but I it’s certainly something I felt in you know, over the last decade would love to get your perspective on that.

Chandra Khatri (13:39):
That’s a great question. In fact, that is the fundamental tool, very close to even us got AI. So I would say the biggest reason for not active adoption is ease of use. So CIOs executives, in fact, sometimes even investors, they know what the problem is, right? Like they know application and end user problem. They know their problem very well. In fact, they know their space and industry very well and technologists who are building, you know, like open source community or, you know scientists and engineers, they are building technologies, but there’s a big gap in the middle. And that gap leads to a not great ease of use a not great experience. So that that gap is leading to not as much or as smooth adoption of this technology. The technology is there actually, you know the current technology that we have today can address a lot of the problems, which CIOs or execut already have just that it is not happening where the base is lower because people in the middle that funnel is missing.

Chandra Khatri (14:46):
Either people who, from the product mindset, they are not getting deep into AI or from PM. And that’s for this, you might have heard of this new roles coming up, which was not there before called technology program manager, technical program manager. This is, these kind of field is coming up a lot and it will continue to grow because there’s a gap between product and executives and engineers or scientists. And this, these are the people who actually understand technology high level, and they also know the application will get to one. So, and that is the, so this is one solution of where things are going. Another one is no code AI. So you might have heard of a lot of these, no code AI companies coming up table bunch of other companies. You know, in fact, even quality. AI is no Codi. What is happening is executives.

Chandra Khatri (15:31):
They don’t need to know, or, you know, the program managers or leaders, they don’t need to really know the AI part of it. They just need to know what needs to be done, or they should need, they just need to know what their problem is. So how to feed the data has been now automated. And, you know, they just need to connect with their APIs, wherever the data is, it’s an AWS or snowflake, et cetera. The second thing is what kind of application do they need? So for that, there are these no code UI tools. So a lot of these no code companies, they just have, they just begun. I would say they will make it much more easy in the next three for five years, things will become even more easy for them. So adoption will be more because now what ha what is happening is the executives.

Chandra Khatri (16:09):
They don’t need to really learn AI. They just need to know these tools. AI will automatically update with time, right? So that’s what even we web AI. So we have this discovery models, as I told, like we automatically build the system and automatically build the bots, what product people or executives or they need to do is they just need to make sure that whatever the recommendations are, those are relevant for their business. Or if some minor changes are there, they can simply do it by UI. Like they can simply make some tweaks here and there in the UI or using some tool, maybe one or two developers, not as much right. To make some connections or integrations, et cetera, not the army of engineers. So that, that gap with this no will get produced. That is from the executive side of program or product side. But from the execution side, this kind roles kind, TPM, TPM, those kind of roles will actually bridge the gap between engineers and the product.

Dan Turchin (16:59):
So let’s build on that. So assuming the quality of the NLP is not the problem. It’s really the quality of the user experience. And assuming there is this new role, the TPM technical product manager exists. What is the optimal user experience that we’re missing today?

Chandra Khatri (17:14):
I would say there, there are some applications which have adopted these technologies more and some application of some domains that has not been the case. For example if you look at healthcare, these technologies could empower healthcare much more, but given the responsibility and given the legal system and also different aspects of healthcare, the adoption has not been as much, same thing with government and same thing with defense as well. Right? So I would say and there’s some domains which are healthy, adaptive, for example, e-commerce right. That’s why you see a table. That’s why you see drag and drop UI or building these days, especially after pandemic building a website, building a business is just a matter of takes, right? You just need to know, matter of, you just need to know your tools. You just need to make sure you’re managing your main, maintaining your inventory via tools like airable.

Chandra Khatri (18:05):
So essentially what is happening is that some domains have been early adopters because of maybe it’s a pandemic or whatever. And also that, that, that, that domains people there is less constraint and they’re more eager. And then they know what they need to do. So people tend to do that with, but other vertical, there has been a less adoption. So I would say, peer question, what will happen? Or what could happen is that the domains which have adopted these state of their technologies, they will guide, and they will also bring up the issues which other domains should be cared for. For example, you know, PII has been a big problem, private, you know, bunch of these things have been becoming problem, even ethical issues. So these other domains will bring those up. So before when we get to healthcare of, or other domains, which are more critical, such as defense and all those, they will, before these applications get there, these kind of problems will be addressed. And by that time, you know, and by the time, once these problems are figured out, they’ll also adopt these technologies.

Dan Turchin (19:06):
So I know you’re on the cutting edge of NLP research, and I saw you’ve done some exploration into a technology, mentioned transformers, explain to our audience, how are transformers being used in the field of NLP and any other advancements in NLP that, that you’re seeing on the front lines of research

Chandra Khatri (19:23):
Transformers as the name, the name is rightly said. So I would say so if I were to explain transformers in layman’s term, it’ll be, I can give an example of pizza, same example you know, transformers, it’s a technology of for, or very well used when you have pre-trained models. So example, you know, I can train lots and lots of pre-train models for different applications. One could be computer vision. One could be let’s say NLP, another one could be, let’s say robotics. So what transformers are great at is you provide lots and lots of data. And, or even within NLP, you can have one vertical could be healthcare. Another vertical could be HR. Another one could be it. So there’s one company who works, right. They’re doing pretty well in the it, there companies in healthcare as well. So in, there are a couple of companies which are building bots and stuff, or NLP applications for health, for HR.

Chandra Khatri (20:19):
So what is happening is transformers is that building block, which you can train with lots or lots of data available or data in that particular application or domain. And once you do that, then you can only fine tune with some samples. So example, if If I take a bot for it, let’s say move works. I can simply not tweak it to company like Nutanix. I can keep tweak it to company like Facebook. I can tweak it a little bit to, for the company like Google. So what is happening is you take this core piece, which is for this domain or this application, and only fine tune for seven end user use cases and that’s transformers. So transformers essentially is the building block for for greatly, for fine tuning of our training. You pre them, and then you use it for different applications on the computer vision side.

Chandra Khatri (21:07):
what transform is the same thing? It’s that you take your gift, train a really good model or transform a security model on identifying objects. So it might be able to identify, let’s say cats, dogs, elephants, or maybe even trees, humans, right. Cars now. But it might not do as well on good. What type of car is it? Universities? Is it is it a Porsche, you know, bunch of other BMW? So what you do is you basically take those transform model and now only give you examples in the end for different you know, Subaru or, you know, it will do all those kind of classification with few examples in and there. So it probably knows that these are cars, but it does not know what brand of their car is. Right? So that’s what transformers do, fasts that technology where you fine tune, train, and good thing about transformer is that it is mostly driven by open source community.

Chandra Khatri (21:56):
So they’re available. All of these models are available. So even if, as a researcher or as a company, or, you know, anyone wants to use it, I don’t need to train from scratch. I can simply take it and only give a few examples for my application. And then that’s it otherwise to get to this point, it used to be six months to one year cycle, which now is, which now only takes a few days, or maybe depending on if it’s enterprise, maybe a week. So what used to take a year or more than a year for enterprise? Now, it is taking only a few days to a week because now you already have this. As I said, pizza based available, I just need to get my toppings and then put those topics on what I like on my application.

Dan Turchin (22:33):
You mentioned PII personally, identifiable information a bit ago, I’m interested as a technologist. What are the ethical issues in and around AI, whether it’s data algorithms, et cetera, that most concern you,

Chandra Khatri (22:49):
I would say the most important aspect is that we have been, or community has been running fast, or, you know, something like guess Facebook has code move fast and break things that has been the I would not say mindset, but I would say rather approach. So researchers because they work in silos or people who build technology, they work in silos and they try to empower as many people as they can. Right. But given that the same technology can, it was impact. Those things are well taken care of. They’re trying to empower the more we try to empower the tools as these tools are more available, people can misuse them, right? So example share how to use chemicals, chemical work with another chemical, all this. Once you make this information very like everyone has access to, and also with tools, people can make bad stuff out of it or something which is dangerous, such as chemical bombs and stuff like that.

Chandra Khatri (23:44):
Right? So point is that they’re more empowered. It’s not that researchers are trying to build those things. Researchers are trying to make everyone empowered. They’re trying to empower everyone, right? But there, they could be used for adverse effects. And that’s what is happening with PI or not PI is one example, but that is what is happening with technology as well. So what can be done is that now that we have built this technology we have, now we know that we are, they have already, we are already at the forefront of where things are and they will be, so now a large group and people who lead these groups, they should come together and they should it would be good for them to come up with guidelines on how to, what to open source, how to open source, or, you know, maybe you want to open source everything, but access of that one could be case by case basis, right?

Chandra Khatri (24:29):
Or maybe the people who are using it, they should be licensed. You know, we have GP license, a bunch of license, right? So that way, you know, people are using this information of these models in a, in a good. Now that, that is the, what I just said is the problem of the application, but not in the generation or building of these models, right? When you build these models, there is other aspects of it. So from that point of view, what researchers can do is they can clean the data in advance, right? Or they can have another set of models, which, which flags the data, which could be biased, which could have PII, which could have other kind of these potential adverse aspects to it. So we can identify those. And then a human in the loop can further remove it or clean it up or whatever, right.

Chandra Khatri (25:14):
Model can do a large part of it. But again, human in the loop can do the final verification and one that then they can start training the models. And that is what is happening these days now, even companies such as open AI, Google, Facebook, they have built large language models, such as three cetera, C what they’re doing is now they have even Google, they, they, they built a large set of steps, or they’ve shared a list of steps, which they account for before this data goes into production or into the model. In fact, even if the model is trained, then there’s a test set, which we can create. And a lot of people are creating email. In fact, we have already ID also create a test set where we explicitly look for whether this model is doing the right thing. Whether is it biased or not? Is it generating any PII? Right. So we have those tests where we try to account for and make sure that this model meets all those requirements. And then only we push into production. So it’s a responsibility of everyone, the ecosystem researchers, and also people who are using and consuming them in applications. So I think, I would say it has to all of us will have to come together and our part to make sure that the application that we are building are safe.

Dan Turchin (26:22):
So let’s say hypothetically guided AI makes a recommendation that causes harm to an end user. They, they use a product the wrong way. They buy the wrong, whatever it is, but there’s actually some tangible harm. And let’s say again, hypothetical, but the data that was sourced by the customer is of poor hygiene. Mm-Hmm biased, it’s inconsistent, whatever the case may be, who is responsible? Is it the, is it the provider of the algorithm, the provider of the data, the system that builds the model? I mean, not that it matters, who’s responsible. Most what’s most important is not harming a customer, but how do you unpack that as a technologist?

Chandra Khatri (27:04):
So then it’s a, it’s a great question, especially relevant to how we do I got at AI, but let’s, let’s try to look at the fundamental problem of how learning works. So if you, if you take kids, for example, kids, their parents, and people around them to try to make things as safe as possible for them, especially you take okay, of let’s say two years or 18 months, right. They’re learning, they’re just trying to explore everything around them. Right? So however checks and balance, which their parents or people around them try to make every once in a while things happen, right? So AI is also like that. It is still developing. It’s not ready there yet, especially in the kind of applications which we are dealing with these days. AI is there applications are there, but still there in the early stages.

Chandra Khatri (27:49):
So what we have to do is to try to give as much feedback as possible, try to make this system as much try to have enough checks and balances. And if every once in a while, when we see that they’re not doing a good thing, then we, we go, then we go, and then we give them feedback. So what we do at guard AI is something similar because you, you sort of brought this up. We, our approach is to discover from historical logs, historical interactions of emails, voice logs, chats, and even articles. So what we do is we discover from the data, the real data, right? So all kind of anomalies would have been taken care of biases would’ve been addressed. And a lot of these real applications scenario might have been addressed largely, but which, which is kind of different from almost every other company in this space where they manually design the flows and manually provide the data, which is very limited.

Chandra Khatri (28:39):
So, because we discovered from real conversations, we address them way ahead in advance, as opposed to when they appear once the application is deployed. So that is one approach that we, that is unique. And that actually address a large portion of a large portion of these issues or large percentage of these issues. And, but every once in a while things go bad or therefore we have this feedback loop. So the moment, if AI did not behave as it was supposed to be, we have this active learning loop, which which learns from the act scenario where it did not do well on, and next time it will try to pick it up and then it will learn from it. And therefore it’ll not be repeating the same mistake. And I think that is what is happening across other applications as well. But there’s a big gap. Not everyone has this active learning loop. And what we need is a good feedback loop, where people or users and users they’re giving us feedback and AI researchers or designers, engineers, they take that feedback and update the system as quickly as possible.

Dan Turchin (29:37):
So Chandra, I gotta get you off the hot seat. We’re bad at time, but you gotta one last question for me. So give what you know about what’s ahead in terms of NLP and kind of broader AI technology. What’s one thing that you think will be commonplace at work. Let’s say in 10 years, you know, we’re having a version of this conversation in 2032, what are we doing then that would seem like science fiction today.

Chandra Khatri (30:00):
I would say AI will be ubiquitous. I mean, it will be teaching students, teaching kids. It will be and the teaching will not end just, or will, will not stop at university level. The teaching will go even at offices, right. Or people will be using AI for coding, for which, or has already started, but not as good right now, but AI will be like a tool. And so in, if If you were to ask me, AI will be a tool which even common people or not, not common people who are not AI expert, even they will be using it. So it will be available to almost everyone in the form of education for learning for their own applications, for their own task, which they do. And that is what AI will. So I would say it is more about this availability of this AI tools that will be very common, which today feels like science fiction, because, you know, we have a, this version of Jarvis in iron man or stuff like that so that we will be getting close to that.

Chandra Khatri (30:59):
The challenge will be how to have single AI. That’s what the problem is. Currently. We have a lot of AI applications, voice assistance built by Alexa or tools such as guard AI building or education. We have different tools will be, there will be different tools. So the common problem which I’m looking forward to is would we be able to bring all of these, all of these things to single assistant or single single AI or single AI communicating with communicating with all of them, that is something which I’m looking forward to, but in general, we will see AI applications everywhere and available to every, every end user

Dan Turchin (31:32):
Chandra. I’m looking forward to that future. That’s that’s pretty exciting. And I agree. I think that’ll be here within the next decade. Let’s see, before I can let you go. Where can the audience learn more about you and your work

Chandra Khatri (31:43):
So they can I’m pretty active on LinkedIn. So I mean, the police free to reach out I’m I respond very quickly and then try to connect as much as I can. And of course there is website where there is contact go AI. You can reach out there, but generally I’m available on LinkedIn. So

Dan Turchin (31:59):
Excellent. This is a fun conversation. Thanks for hanging out.

Chandra Khatri (32:03):
Well, thanks Dan for the invitation and I really enjoyed this interaction.

Dan Turchin (32:06):
You bet. Well, that’s a wrap for this week on AI in the future of work. I’m your host, Dan Chen, but we are back next week with another fascinating guest.