Lisa: Welcome everyone to today's "Business as Unusual" webinar. We are so excited to have you join us for our last session of 2021. It's hard to believe that it's already that time of year but as we end the year we are excited to have our guest speaker today joining us to walk through how we can start 2022 off with some amazing intelligence and knowledge regarding the topic of AI, so we'll be getting started here in just one moment for those of you joining us via LinkedIn and Facebook welcome as well we'll be pulling you in here just momentarily so as we get started give me one moment folks, and we will get our live-streaming service up as well perfect we should be having our guests joining us again those of you via Facebook or LinkedIn welcome to today's webinar as we get you up and running as well make sure you've got a pun and paper or remarkable two or your phone handy you'll want to take notes during today's session our guest speaker Niko will be sharing so much great content with you, we were joking beforehand you're all going to be data scientists by the end of today's session well with that let's go ahead and get started so again welcome to today's "Business as Unusual" webinar where we are speaking with industry leaders who are helping us survive and thrive in today's economic climate my name is Lisa Behning, I'm an account director at Red Caffeine, and I'll be your moderator for today's session as I mentioned before we are so excited to have Niko people off joining us today Niko is the founder and CEO of hybrid intelligence and what we're going to cover with him today is really talking about your business problems but framing them up in a new way on how you're building teams to help tackle complex problems we're going to be talking about machine learning with AI, so why should you care about that what are the learning opportunities that comes with it what are the qualities you should be looking for a team as you look through machine learning and the type of people who are going to help you solve these complex problems, and we're also going to talk about this clean one-room approach that's literally what we have to help you make innovation scalable so a lot of great content in today's session as a reminder for those of you who are the first to register for today's webinar we do give out five twenty dollar Grub hub gift cards to the first five registrants, so I'd like to thank Karen kaya colleen l Gail n tony b and Sarah p for being those first five registrants you should have received that invitation yesterday if not please go ahead and reach out to us at Red Caffeine, and we'll make sure you get access to that as a few housekeeping notes we do have everyone on mute today so for those of you joining us we assume please use the q a or the chat option to ask questions I'll be monitoring throughout and be sharing those with Kathy and Niko, so they can answer them as time allows and at the end as well for those on Facebook or zoom I encourage you to also use the comments section I'll be moderating there as well and feeding those questions over to Niko and Kathy so with that let's go ahead and get today's session started, so I'm happy to introduce my co-host Kathy Steele who's the CEO of Red Caffeine as well as our guest speaker Niko pipola Kathy and Niko welcome.
Niko: Thank you so much; good to be here.
Kathy: Thank you, Lisa. Yes, Niko, we are so thrilled to have you. This is a topic that we're all excited about, learning more about, and thank you for joining us today, Lisa gave a little bit of a backgrounder on you, but I'd love you to kind of step our audience through what you do at hybrid intelligence and a little bit about your career journey.
Niko: Yeah, sure, so I started my background in technology. My education is in computer science. I have a bachelor's in computer science. Started my career kind of in an i.t realm but very quickly moved into technology consulting. There was a company called diamond consulting that was very technology-focused kind; of gained my basic consulting skills there and then eventually moved over to PricewaterhouseCoopers, where I was one of the founding members of our emerging technology lab; essentially, this was a group that was dedicated to building new skills in emerging technologies and then applying those to help our clients do the same it was a fairly unique endeavor it was the only group of its kind, but that's really, where I learned a lot about how to set up and run innovation groups and then in the last three to four years of my career I've really honed my focus to a particular technology that being machine learning and artificial intelligence.
Kathy: Excellent well we always try and tee up how this would associate to growth, and technology obviously is a huge growth lane to invest in, and I think it underpins all the business functions and operations today sales marketing ops finance and HR, but when we're thinking about artificial intelligence and machine learning its doing as big of things as helping to detect cancer , but it's also really starting to be seen and integrated into manufacturing applications preventative maintenance equipment failure , and then we're seeing it also in our everyday life and in using tools in our home like Netflix the streaming applications that we watch TV on today Alexa those types of things, so I'm really excited about sounds very lofty and big in some cases, but I'm excited for you to kind of break it down and help us see the opportunities we all have in our business so with that I know that AI is predicted to reach about 98 billion by 2020 in terms of corporate investment, but it has been a little bit of a slow adoption rate for most businesses, so I'm hoping you can kind of set the table for us on why is corporate innovation something that businesses should be considered considering in their investment strategy for growth.
Niko: Well yeah, so I guess let's parse those two things out because there are those are two kind of related but different topics corporate innovation and then the particular technology of machine learning and artificial intelligence and just so the audience knows I'm going to use those terms interchangeably learning is actually the more accurate technical term most common like nomenclature uses AI, but they're really the same thing they're referring to the same thing so why innovation why isn't innovation important well I mean I don't know if the folks have noticed, but the world is changing faster and faster, and it's becoming ever increasingly more difficult to keep up I just think it's paramount I don't think a business is going to survive if it can't learn new skills and apply them rapidly it's well I mean we're just seeing technology basically transform industry after industry, so I'm going to be talking about one particular type of innovation strategy that being an innovation lab that's where I have the most experience it's not the only one , there are a know you, can do innovation through mergers and acquisitions there's other corporate partnership programs there are lots of different ways to try to do this , when you look at a more traditional company a company with kind of more of a legacy sometimes you have these large-scale digital transformation efforts consultants like myself will come in and say we're going to modernize your entire technology stack overnight and historically these have not fared so well it's just introducing that much change at one time there's usually a lot of resistance in the company because of entrenched interests and also of course you need to upscale a lot of folks so as opposed to trying to change everything at once one other approach is this idea of an innovation lab, and it's kind of like planting a seed it's a small group a small part of the organization, and then you allow that to kind of expand from there , so just looking at the slide what are we talking about here so it's not just a brand first it's not just like oh we're going to just keep doing what we're doing, and then we're going to call part of it this new thing it's important that it actually does have some very distinct properties that are different from the usual technology department within a company, so digital innovation lab let's break that down, so we're talking about digital technologies here so usually software in some cases hardware, but I don't have experience in material sciences and biotech those are different types of innovation labs that you might find that are much more specialized so just letting we're not going to talk about that kind of stuff when we say innovation we're talking about an innovation funnel what that means is you have a well you have a funnel with a lot of ideas coming in on the left-hand side and there's a series of ways to filter and prioritize those ideas and then only very few of them most will often make it out of the end of the funnel and actually get implemented into the business , so you have to be comfortable with the idea that you're going to try things that are going to fail, and we'll talk a little bit about metrics and ROI and what the impact is there and then when I say a lab I'm talking about a specific environment most often a physical space obviously with the pandemic that can be a little harder, and it can be done with a virtual team but most often you want a dedicated team of resources that are sitting in one physical space and the environment is specifically designed to remove friction and barriers for trying new things when I say the environment I mean the the charter of the group what they're at what their what their mission is the skill set of the team but also the technologies they're using.
Kathy: So, almost like a little entrepreneurial company within a company kind of group is out, and so how do you really get started on something? How does a company start this type of endeavor?
Niko: Well, I mean the first thing would be what is it that you're, I guess defining the goal of the lab like what is it you're trying to learn what is it you're trying to develop and based on that you would hire a certain set of skill sets a certain skill set of folks on the team and then find a dedicated space for them, I mean it's we started basically in a closet to be honest when I was doing this at PWC they basically put us like in a windowless room it doesn't have to be anything fancy, but you do need dedicated people with a technology stack which is separated from the rest of corporate i.t, and we'll talk a little bit about why that is.
Kathy: All right, well, let's step us through the process that you're recommending.
Niko: Okay. Yeah, so here's a kind of framework I had developed so after running our innovation lab which was based out of Chicago for many years we had a lot of other clients reach out to us as well as other parts of the organization globally and ask us basically what you're asking how do we get started and what are the components and so this is kind of a framework that I had put together , so I had mentioned the two main pieces here which is the lab environment, so this is the space the technologies and the people do you actually need to run the funnel and the funnel are what you use to learn new things, so you have the lab environment you set that up first, and then you're running projects through the funnel so if you jump to the next slide we'll go through each of these in turn okay so foundation so budgets it's important that there is a dedicated budget for an innovation lab which spans at least a couple of years you two or three years minimum the thing with innovation projects is many of them fail right so if you are determining the success of the lab based on the first project you do you might end up shutting it down before it can actually bear fruits innovation projects are unpredictable, and you have to look at the ROI is still important you would never I would never propose that a business do anything without an expected return but that return should be measured on a portfolio basis over a longer period of time, so individual projects may succeed may fail the portfolio projects that should deliver a positive ROI over the course of a couple of years and this is where also executive support comes in often if you don't have a c-suite sponsor for this kind of effort it ends up getting shut down because budgets especially when budget is tight this is going to be competing against production level operational production things and , and you need someone to kind of protect the group while it's developing and learning and eventually bearing fruits does that make sense.
Kathy: Yeah, absolutely, and I mean, I also believe it's just, if you don't have dedicated resources, something like this, think about it as an investment strategy versus output of, and actually categorize it as an investment in how you look at your overall finances for your business I do think you'd be set up to fail the window would be too short and you wouldn't be able to get that experimentation needed to build that foundation so yeah absolutely.
Niko: Yeah yeah, it's talking about like the skills you need and the metrics; it's a lot of these things do tend to get shut down in kind of tough times so to basically to have a longer time frame as you're saying.
Kathy: But, we talked about, when we were prepping for this, sometimes you can start off, with a fairly small group of people, you could maybe recruit some talent that isn't as high cost as real experience talent, so talk to us about the people side of the investment.
Niko: Yeah, actually it's an it's an asset to start to be small you don't want to be a huge line item on the budgets we started with four people and , and we had to get very creative on how we were recruiting because I was this was in the context of a consulting company not in the context of a software company so when it came to getting good talents as front-end developers back-end developers data scientists etc. we what we actually ended up doing was creating a tour of duty program where we would just reach out to literally anyone in the company that had a passion for technology, and we would offer them to join our group for three to six months we would basically train them on the skills they needed they would work on a particular project, so the group was called the emerging technology lab, so we would pick an emerging technology like it could be machine learning it could be an internet of things it could be blockchain virtual reality we worked on all the different things, but essentially the only requirement we had was that folks were super passionate and were willing to put in the time and nights and weekends to get up to speed sometimes we got lucky, and we found folks that already had development experience but in many cases they had never done any developments at all.
Kathy: Wow, wow, so really, really, totally innovative learning on the job, that's pretty cool and exciting but.
Niko: Yeah.
Kathy: We always talk about career-building opportunities too, and so sometimes when you're not in a job that you are in a management position, these are the types of things that you do to improve your own the portfolio of skill sets, so what about the technology piece what are some of the things that are in consideration when you're thinking about the enablement piece.
Niko: Right, so the entire purpose of the technology stack is to enable rapid prototyping and remove barriers to do so the technologies that are good for those two things are vastly different than the technologies that a IT department would use to run a scalable secure technology stack that runs the business operationally right, so it's important to create a clear line that this is not i.t right it has a very important charter to protect the company there are a lot of security requirements so like let's say I wanted to try a new open source library it might say well it's like we have to do a security review there's like a 100 point checklist you have to go through that makes sense for their environments but you that that kills innovation like I need to be able to try three different libraries in an afternoon right to be able to just see what they do see how, and I'm going to break things all the time so the in order to protect the kind of the risk associated with that it's important to put an innovation lab within its own sandbox I mean technologically separated from the rest of the company so that means dedicated connection to the internet dedicated computers it should not be on corporate it's domain.
Kathy: Okay, so truly like its own company infrastructure. I mean, that's pretty, yeah.
Niko: So that way you can play around, you could try things, you can break things, and you're not going to be taken down the company website.
Kathy: And, we have seen that.
Niko: Yeah.
Kathy: So, once you enable this digital innovation team, you built that foundation, you've got your people, you've, parse out how you're going to go at it from a technology stack standpoint; how does this get applied.
Niko: Yeah, so why don't we jump to the next slide we can talk about the kind of the funnel so the idea being listened is like how do we gather ideas of what to work for what to work on rather learn is how do we prioritize and then prototype those ideas and then apply is how do we push those learnings and lessons and maybe even products out into the company , so I'll go through each of these so as far as how where ideas come from for us at least it was two-fold one was capability driven so what was realistic for us to do like what capabilities do we have for instance self-driving cars are super interesting right as a team of four we didn't have the capabilities to build a self-driving car it just wasn't realistic so like realistically you're our small lean team and what capabilities you have in house to actually develop it and then also what are your customers what's top of mind for your customers right is it machine learning is it internet of things is a blockchain where's the biggest impact for your customers, so I mean feasibility and impact really is the two things and that's how I also would prioritize basically we'd listen to customers we look at the technology landscape and see what's happening and then prioritize based on feasibility and impact and then importantly build working prototypes so especially coming from a consulting environment there is this idea that consultants generally build decks right we work with Excel we work with PowerPoints at least for this type of group most people learn by doing, and you don't really learn the true limitations and capabilities of something unless until you try and so developing a working prototype is critical to that in doing, so I mean that's really where the learning happens in doing so you will learn let me take machine learning as an example when I talk to folks about machine learning I usually get one of two responses either they don't know what it is and have kind of dismissed it, or they think it is like the second coming and will cure all the world's ails once you start working with it that'll cure you of those two ideas like it is there is very serious constraints on its on what it can do mostly related to the data you have but in certain environments it's extremely powerful and and so you by actually working with something of course you get those insights and then finally apply, so I talked so much about the importance of creating kind of the separation between this group and the rest of the company this is where if all the lessons stay in this group you're really not getting any of the advantage you have to then find ways to push everything that you've learned back out into the company, and it's important that anything that this group would create it would not support once it goes into kind of production environment, so this is very much a prototyping group if something was explored or developed which folks think could be applied into the larger organization it needs to find a home in the large organization and so there's going to be like a handoff there.
Kathy: So, honestly, it reminds me a lot of the manufacturing process where you're, starting with a proof of concept or first article or a prototype of something, and usually it's coming from somebody else, and then to try and get it to manufacturability it definitely sometimes has to be retooled and tested and re-engineered so pretty interesting I how tied to the listener or in the listening part of the funnel are you looking at like the business overall growth objectives and trying to find a way to align with where the business is heading in order to really get that C-level buy-in because it seems like though those kinds of opportunities if they're aligned with you, know the vision of the organization's growth it would be a lot more feasible to get the investment and get the appropriate other pieces to the finish line.
Niko: Yeah, absolutely, there's a piece that I actually forgot to speak to. It's nice; it's good to have an organization like this, with some sort of executive advisory council kind of on top which could check in once a quarter or even once, twice a year or something and make sure that the innovation lab is tapped into the strategic growth of the business and where it wants to move to the right, so you would expect someone like the CIO the CTO perhaps a digital officer to be on that council and because they're going to be they're going to have the ear of the CEO and the CEO, and they can basically help guide this group to particular technology opportunities that are aligned in that way.
Kathy: And then in the application process, that feels like implementation testing, and is there a number of like feedback loops baked into that process so that you're getting, or you've got a time frame to evaluate the performance of that new innovation and.
Niko: Well, so testing and performance those are concerns for a production application, and you would need a technology team outside this group that would be dedicated to owning that, so this group, after it's proven something useful, I mean, of course, it could be different from company to company, but the way I've seen it work best is it than the kind of does a handoff to a more dedicated technology team because this group now has to go learn about the next thing that's around the corner.
Kathy: So, somebody else taking it to the next stage of development is kind of the way, okay.
Niko: That's the idea for something that's moving into more production phase, yes.
Kathy: Great, all right, so, in these types of scenarios, is there a certain size of company that is, is their size limitations to organizations that from your experience.
Niko: I have no, I mean, I mean, I've I worked at, a very, very large company 26:21, and my last client was a medium relatively medium small company 26:27 around 600 folks I think a startup 26:33 like you wouldn't do this in like a 10-person team because then it would just kind of be the whole team but 26:38 I think medium to large organizations probably would take the most advantage here I haven't, no I mean I've I worked at a very large company, and my last client was a medium relatively medium small company around 600 folks I think a startup like you wouldn't do this in like a 10-person team because then it would just kind of be the whole team, but I think medium to large organizations probably would take the most advantage here.
Kathy: Okay, and then like, so we've talked about how you kind of get there, how you form this; let's step into the machine learning aspect, so it talks through this.
Niko: Okay yeah, so, well quickly why are we, how are these two topics tied together so , machine learning is just a technology of particular interests these days I mean my career I had focused on lots of different technologies, but ultimately I found this particular one to be more important and more impactful than anything else I worked on and that's why I decided to focus on this so if you wanted to develop machine learning capabilities in the company an innovation lab could be one way to do that and of course there are others so okay what is machine learning anyway let's start with an explanation of what traditional software is okay, so traditional software there's some sort of input data and then I as the human developer the software developer I have to come up with the if then else statements the logic gates that would take that input and convert it to some output and that output could be a number eight output could be something that goes pops up on a screen that output could be something that moves a machine right, but essentially it's up to me as the developer to figure out what are what is the particular algorithm and the particular logic gates if this happens then do that else do something else to convert input to output make sense.
Kathy: Yeah!
Niko: Okay, the way that machine learning is different is that it flips that logic it says if I give you a bunch of input data and I give you the output data that I would expect from that you the machine learning algorithm figures out the logic gates, so there are certain types of problems where it's just not feasible to come up with all the IFL statements so like if I give you a picture of a cat it's just a bunch of pixels and I say okay define all the rules such that I can give you any picture of a cat, and you'll tell me whether yes it's a cat or no it's not that it's the complexity is overwhelming you just can't do it so instead what we do is we say here's a thousand pictures some of them are cats some of them are not cats we will give you the correct answer for each of these pictures here's a picture it's a cat here's a picture it's a cat here's a picture it's not a cat the machine the power of machine learning is that it figures out how to map a know input to output such that once and that's called training it so once you go through that process, and you've trained it now I can give you a new picture where I don't tell you what it is because I don't know what it is and the algorithm could hopefully if it's doing its job right with high accuracy predict whether that is a cat or not.
Kathy: Yeah, I mean, I guess, I think it shows up in a sort of really simple forms like chat where you're actually chatting to a bot, and it's fielding through some questions and based on what it's learned, and then it will transition you to a human if it gets to a point where it can't answer your questions automatically and or like Alexa you might be searching for something for Alexa but if you're listening to some types of music Alexa will start to suggest things for you, so I think those kinds of use cases sort of simplifying the different business applications or personal applications that probably pretty much experience on a daily basis now so what are some of the most common things that you've seen in terms of that businesses are investing in.
Niko: Well, you brought up some really good examples I will say that if you have very highly structured data so like say text data in a database categorical data more traditional techniques are it can be used machine learning is extremely powerful with non-structured data, so media like video audio images all of that stuff kind of it kind of got unlocked by machine learning before we really couldn't do much with it, and as far as what are the type of things businesses can do well I mean you're essentially trying to find patterns hidden patterns in the data and so the use cases are pretty varied actually yeah in this slide we have a couple here at the bottom so well let me speak to what we're looking at here so there are two major categories of machine learning we have supervised learning and unsupervised learning I will say the names that have been chosen for all of a lot of the things in machine learning are super confusing they've come from a lot of different disciplines , but essentially in the example I gave earlier with the cats that would be supervised learning means I'm going to give you a lot of data, and I'm going to train that data with the correct answer right, so I'm going to give you a picture, and I'm going to tell you this is a cat we're super in a sense supervised because we're saying like it's like if you were to take a child, and you would say you would point to something and say that's a cat right you point something else and say that's a dog like you're training the child to understand the pattern right that's supervised learning there are two different kinds and this is dependent on the the type of thing you're measuring whether it's continuous or distinct so a con like regression models the answer is a number a continuous number so like I might say predict for me the the price of a certain house let's say I i train a model where I give you the square footage I give you the number of rooms I give you the number of the number of floors, and then I ask you to give me a price because that price is a continuous number than that's a regression problem first the classification problem is given the number of rooms and the square footage tell me whether it's likely to be a house condo townhouse or apartment right I've defined a set of categories I'm classifying it into distinct categories the reason these things are separate is because, the particular techniques that you would use for one or the other are very different, so it's important at the very beginning to understand what is it you're trying to know, and you may be asking one question around a problem that's continuous, and then you might have another question that's categorical that's fine, so you have to create two different models for that so back to your question what can you do with that so here are some examples predicting real estate prices stock prices' employee churn , so spambots are one of the earliest uses of a classification problem basically we're training a machine to say is this spam or is this not spam right two categories two distinct classes predicting employee churn would be like is this employee likely to leave the company or not right you can ask that question in a binary way which would be a classification problem, or you can say what is the percentage likelihood that employee will leave which is more of a regression problem right that's a continuous answer.
Kathy: Yeah, that's perfect. I was already thinking, the real estate like Estimate, where you're getting an expected estimate for a price versus like an apartments.com where you're actually giving feeding it requirements radiation versus classification and those two situations but very interesting so in terms of unsupervised what does that look like.
Niko: So, unsupervised learning is, it's a set of techniques where you're trying to find the inherent pattern in something, but you don't know like I'm looking at clustering now, so, imagine customer segmentation. Imagine I have a bunch of customers. I don't know how many clusters I should have, maybe five, maybe 20. There are techniques that I can just say look at the data and tell me how things generally group together. I'm not going to tell you what the groups are; I'm not going to tell you how many there are but find inherent patterns in the data such that like is similar to like, so this is where you don't know what the groups are, but you want it to group based on similarity.
Kathy: Right!
Niko: Okay, does that make sense?
Kathy: Yes!
Niko: Social network analysis is another example listed here. So, like if let's say we're Facebook Meta now or whatever they are, and you just say, okay, find people that are similar to each other or find people that are closely connected right, but I can't, I'm not going to tell it ahead of time there's going to be 150 of these groups like the data should determine that how many groups right, so that's clustering.
Kathy: Yeah, I would think about it in terms of advertising and trying to target the right people through that type of analysis, yes, right in front of people.
Niko: A couple of things I will say, the limitations of these things because you think like oh what wow regression can do stock price prediction right like this is just the money-making machine all of these algorithms are limited by the quality of the data you put in, so that's the first thing to say also they're always working on information from the past right so black swan events or just other changes in the environment changes in in in patterns machine learning algorithms are assuming the future is similar to the past which sometimes is true, and sometimes it's not true.
Kathy: So, I know Lisa mentioned that you were going to take us through some real-world examples. I think you've got something that a case study that you're going to talk us through but tell us beforehand what a typical project could look like.
Niko: Yeah, well, this is like this list. What is it seven different steps from generating a hypothesis to deploying a machine learning model actually the only thing I want to really emphasize on this slide is all the arrows that point backwards essentially it's a very iterative process and this is different that traditional software developments it is really more of an art than a science it's you kind of like start with a bunch of data you try a certain set of algorithms you see what the results look like most often the first time around the results are terrible you then start tweaking you remove some data you add some data you combine some data you create new features based on existing data you run the algorithm again maybe it does a little better maybe it does a little worse but in each case you're iterating and iterating and iterating, so it's very difficult to predict from the outset okay what data is going to be relevant and what accuracy are we going to get right it is very much a kind of just a process of discovery which is different for difficult for some folks especially executives that are looking for give me a guaranteed result before we get started.
Kathy: So, is there any way to like fast track the evaluation process because I know we've built things like this as well where we're trying to give somebody some selection tools or criteria, mostly in that shopping formula yeah, how can you test this without putting a human on testing each feature and function to make sure we're getting good outputs or is it really going to require a lot of human testing before deployment.
Niko: So, in some sense the problem is computationally irreducible what that means is there isn't really a shortcut like you have to kind of do it to see, but there are ways that people are fast tracking this essentially doing running a lot of different algorithms in parallel and that is there's a whole suite of tools called auto ml auto machine learning and it kind of brute forces the problem in the sense of like one of the key decisions that needs to be made is what type of algorithm, so we talked about the different categories in machine learning given any one of those categories let's say regression there's probably at least 25 different algorithms that you could use, and they work better or worse and different with different data, and different environments so going through those one by one is painful and tedious, so there's been a series of tools where they just run through all of them they just feed your data through all of them compute power is relatively cheap now they brute force the problem and then outcomes whichever one performs best so that is a very common technique to try to move faster as far as the other decision to make is what data to use so more data generally is better, but there are a lot of dependencies with the algorithm with that as well and of course bad data negatively impacts your results so the cleaner data you can get the better there is a there is a step called where you're doing feature engineering and future cleansing that's probably getting a little too technical for today , so I'll kind of leave it at that.
Kathy: Yeah, I mean, I think this does sound extremely complex, but I kind of go back to your initial, sort of why invest. I mean, this is mission-critical for most organizations to do something, if not at least deploying some types of technologies that are already used like machine learning features to help them automate or fast track certain business functions, so talk us through the case study and me, and so I think people can get their arms around like what's possible and how to scale.
Niko: Right, okay. So, I had a client, the CEO who attended a conference decided that AI and machine learning were important for them to learn. This is a very traditional company. They've been around for over 100 years, manufacturing space roughly 500 to 600 people. They had never done any sort of data science before they had because they've been around for a long time. They had a lot of data, but they never had. They didn't have many folks on their team that knew these tools or new data science, so they reached out to me and said, how do we get started? Ultimately it was the innovation lab kind of approach that I recommended, and that's what we tried. The first step was that we did two things well in parallel one was setting up the technical environments. So, because they didn't have any experience with this, they didn't have the right tools to do data science, and so they didn't want to buy a bunch of servers as a capital investment like that. So, we set up a cloud infrastructure for them right so using AWS basically set up everything in the cloud to get started was free, and then I think the first month was like a couple of hundred bucks okay this is a drop in the bucket for an I.T. department, and then we needed talents, we needed skills ultimately we ended up going through one of their university connections and finding some really talented young folks from the engineering department of a university and that was enough to get us started we brought him in for just for the summer for like a five-month period and well it was a small team, but we had a team all right and the next problem was actually also in parallel but what's what to focus on what to try and there was a lot of discussion back and forth about okay we want this has to be a big impact problem we have to like change the company here, or it has to be , low-hanging fruit something that we know is going to be a win and often those two things are in conflict you can't have both ultimately we looked at where the company has good data because again for any machine learning project it is garbage and garbage out if you don't have quality data that is your first step, and you really won't be able to achieve much as far as the algorithms are concerned, so we knew that they had a lot of good historical data around their clients and their customers and so that's the area that we focused on ultimately we settled on a type of problem which is called lead scoring the basic idea is if you have a lot of information about prospective customers like people that have reached out to you and sent a request for a quotation or something, and then you have information about which of those RFQ is converted to orders then potentially you could develop an algorithm to predict whether a particular request for quotation will likely convert into an order and this of course will allow you to allocate resources and do market segmentation more effectively right there might be areas where you're consistently not winning the work and you may want to either divest from that area or place more resources those are business decisions that have to be made, but ultimately that would give you more information around that and there was I mean so that was the problem I mean we the biggest challenge honestly was around data quality there were a lot of data quality issues that we discovered through this process that they didn't know existed, so they had been working with some bad data in their business, and they just had no one looked no one had ever looked, so they didn't know , but it came out in the algorithm because we would we would we would run it we would run the numbers and either the results would be really bad in which case we knew something was wrong with the data or in some cases even something worse would happen which is we would get near 100 accuracy which sounds great, but that is very suspects something you're usually thinking okay something's going on here and what was going on was that there's something known as target leakage which again I i think might be a little too technical to go into in very detail, but you have to be careful that you're not training the algorithm on information it shouldn't have at the time it's making the decision we were including historical data which was updated later on in time and the algorithm was using that.
Kathy: So, in terms of how it worked in their business, were you able with it? Has it gotten to the deployment stage yet?
Niko: So, they have the algorithm, but they are currently in the process of updating their system that would not update and completely redoing the system that would essentially be used that this algorithm would be embedded in, so this model, as I understand will be included in the new system, but they're kind of going through that right now.
Kathy: Yeah, but I mean, I just love it because it sounded really lofty when we started, and you were talking about how to get this underway, and then it really got to a real tangible opportunity for most businesses to solve a pretty important problem around where they're spending their sales dollars and their sales resources so lead scoring I think is a hot topic for many organizations, but I would assume that it would eventually get you to know somehow integrated into their CRM system so that they could use that in one view one single source of truth so but that I think, I think maybe people that just start us thinking about this type of investment are breathing a sigh has released that you can do some pretty straightforward projects, and possibly leverage talent that is like emerging and wanting to learn some new things.
Niko: Yeah, real quick. I mean when, when we look at the actual costs of doing this, we did it with super cheap cloud infrastructure and interns.
Kathy: Yeah!
Niko: You can get started really easily.
Kathy: Yeah, I mean, I know we've worked with universities on some of these research projects similar to this, but yeah, that I hadn't really thought about tapping into a team in this manner, but it's going to be now on our radar. So, thank you, Lisa; how is there anything coming up or anything we've missed?
Lisa: So, I think Niko, one of the questions we received from our zoom audience was, are there any companies out there that might be well known for doing this. So, if I were someone getting into this, is there someone I could emulate? So we obviously know you have a partnership with PricewaterhouseCoopers, but anyone else in the market that I could say wow gold standard or other industries even that might be highly using the machine learning that I could look to.
Niko: Okay, so well, as far as what companies are using machine learning, I mean, every technology company is using machine learning. I mean, I'm not going to list them because it's just like, what company isn't using machine learning? As far as what consulting companies can help you get started, my company can get help to get started. Obviously, hybrid intelligence LLC reached out to me. As far as what's small, sorry, what was the other group?
Lisa: Yeah, just if there were any companies out there, so obviously it, but have you seen any other manufacturing companies, retail companies, other people that somebody could go, oh wow, oh, they're using, machine learning that maybe I wouldn't think of.
Niko: Oh, man. It's honestly, so ubiquitous, so any company that is sitting on a lot of data and that could be structured data like in a database it could be audio logs from call center information it could be a video I mean a lot of media or structured data, and it's trying to identify patterns in that data and traditional they've exhausted what traditional analytical techniques can do could look into machine learning as a tool.
Lisa: I love hearing more of the media piece. I think everyone, too, especially in sales, would instantly go, well, we've got a CRM, and I've got names, and I've got data, but this can be truly anything as long as you've got quantifiable to be able to start reviewing and teaching.
Niko: Right, Right.
Lisa: All right, so the next question we got from our audience, and I will say this was the second question that we had when I had done our intro, I had mentioned this clean one-room approach, and so someone asked like I haven't heard it brought up, yet so could you explain more about what that concept entails Kathy maybe help out I know we had that in our notes, so I had used it as our insurance today.
Kathy: Yeah, I know I don't. I don't recall what that was about, so sorry. I can't help.
Nika: That's not mine either.
Lisa: Okay, I don't know where that came from. The copywriting team came up with as far as the process going through because it sounds like it is methodical when you are going through, but as you said, there are a lot of feedback loops as you're going through to make sure that the process is completely happening.
Nika: Well, let me say this I think probably what that's referring to is this idea of not being able to effect change across the organization at the same time everywhere, and so going back to this idea of innovation lab being the one-room perhaps that's what they're referring to where you're incubating new ideas new ways of working and then as certain things work then you're slowly kind of encouraging them pushing them out to the rest of the organization.
Kathy: Yeah, we probably. It was probably something from our prep session that was just.
Lisa: Really, that phrase stood out. Yeah, obviously, we made an impact on our internal team so well as we are going past, and thank you for everyone staying a little bit past our typical 45 minutes. I'm going to go ahead and share Niko's contact information, so for those of you who may be watching live or perhaps watching our recording, if you have questions for Niko later, you can reach out to him on LinkedIn as well as via email; just make sure to include "Business as Unusual," so Niko knows where the questions are coming from Niko thank you so much for sharing this I think truly, as you said there's so much to learn about the idea and the process of data science, so I don't know if we're all data engineers yet.
Niko: Pretty close. This is my pleasure. Thanks for having me.
Lisa: Well, as we look ahead to next month as we kick off 2022 we will be starting on January 20th with our first "Business as Unusual" webinar of the year we currently have a duo tiered up for this Claire Ellerhorst and Michael Rary, so Claire is the assistant VP at fifth third bank Michael comes to us from capital strategies, and they're really going to be diving into the trends of 2022, so we all know 2020 obviously looking the past very hard to predict had ripple effects into 2021 but what's ahead for 2022, so they're really going to be discussing how the trends which ones you should be watching for how is that going to impact business growth and really giving you that pulse for 2022 and what it's going to do for today's competitive business landscape you can go to redcapping.com to register we'll also have this up on LinkedIn as well, so you can join via LinkedIn events also looking ahead to January 2022 is Red Caffeine's annual people plus people conference we've had to postpone our date due to covet, but we are excited to be bringing this to you on from excuse me January 27th which is a Thursday Kathy I know you're spending a lot of time getting ready for this event in just a few short weeks anything you'd like to share with today's audience.
Kathy: Yeah, I think it's going to be an action-packed day but a really interactive day, so we'll be learning from our presenters just as much as we're going to be learning from our sponsors and our attendees. There's just going to be a lot of business insights shared, and we really want to have this be a really interactive session, so we're excited to be getting back to a live event format, keeping it to one day so that we can get a lot of people to attend it's a short commitment, but I think you'll have things you'll walk away with ideas that will really take you'll be able to take and put forth in your business growth plan for 2022. So, we're, so you can sign up at peoplepluspeople.com.com and so looking forward to seeing everybody live in January.
Lisa: And, as you'll see when you go to our peoplepluspeopleconf.com website, you'll see a bad hat that ass doesn't happen overnight, but it can happen in a day, so please join us for this one-day event with all the amazing content and just getting to meet in person with other fellow like-minded like-hearted business leaders. All right, as we close out today's session, thank you for joining us for our last "Business as Unusual" webinar of 2021; you can always go to our Red Caffeine website to watch any of our past webinars that have been started since April 2020 a lot of great content there. So, as you're heading possibly into the holiday break and looking to amp up your business knowledge, definitely check out the red capping website for more resources; if it's your first time listening to a "Business as Unusual" webinar. We thank you for joining us; if you'd like to hear more on how Red Caffeine's growth consultancy approach can help your business, you can reach out to us at connect redcaffeine.com. Thank you so much for joining us today, and everyone enjoys the rest of your day; thanks so much.
