Data, Machine Learning, and User Experience, with Giorgos Zacharia, CTO, Kayak

Giorgos Zacharia Leading the global KAYAK Technology and Product team. Overseeing the company's global technology strategy and product development for KAYAK's websites and mobile applications. Responsible for all aspects of Product Management, UI Design, Technology, Engineering, and Data Science.

43:21

Mar 13, 2015
1,414 Views

Giorgos Zacharia is Chief Technology Officer (CTO) of KAYAK. He joined the company in 2008 as Chief Scientist. Prior to joining KAYAK, he was co-founder and CTO of Open Ratings, Inc., a leading supply risk management provider. Giorgos also co-founded two machine-learning driven hedge funds, Stocknomics and Emporics Capital Management. He holds four math and computer science degrees from MIT, including a PhD, won five medals in the International Mathematical and Physics Olympiads and was a Fulbright Scholar.

Kayak CTO, Giorgos Zacharia: Discussion Agenda

Background

  • Tell us briefly about your professional and academic background
  • Tell us briefly about KAYAK?
  • What is the role of CTO at KAYAK?

Data, Analytics, Customer Experience, and Product

  • What does KAYAK actually “sell”?
    • Data, information, user experience, or something else?
  • Given this, what are the core competencies of KAYAK as an organization?
  • Give us a non-technical overview of how data flows from suppliers, through KAYAK systems, to users?
    • What are the primary technical challenges you face?
    • How do manage such large volumes of data?
    • What is the link between data and user experience?
    • What is the role of predictive analytics in the KAYAK product?
    • Explain the role of machine learning to KAYAK?
  • Which is more important, technology or user experience?
  • How do you personalize the experience for individual users?
  • How do you optimize the website for traffic?
  • As a travel site, mobile is obviously critical to your strategy
    • How does mobile fit into the product landscape?
    • What are your guiding principles for mobile design?
  • What are the most challenging parts of the overall value chain we have been discussing?

Business and Culture

  • What is the composition of KAYAK’s employee base?
  • On your LinkedIn profile, you specifically tell potential candidates to find a referral. Why?
  • Does this approach enable you to hire sufficient numbers of data scientists and other specialized experts?
  • What kind of culture is needed to support both intensive data science and highly engaging and friendly user experience?
    • How do you maintain that culture?
  • What approaches do you use for getting user feedback?
    • How is customer success linked to company culture?
  • KAYAK obviously relies on continuous technical innovation
    • How do you maintain that pace of technology change and advancement?
    • Do you have formal innovation processes?

Closing Thoughts / Advice

  • What advice do you have for startups trying to build a data-intensive company?

Transcript

Michael:         

(00:02) Hello, welcome to episode 102 of CXOTalk. You know, data, analytics, machine learning these are hot topics and today we’re going to be talking with somebody who is truly a leading expert at applying these concepts to the real world to travel. I’m Michael Krigsman and I’m here with my co-host Vala Afshar. Hey Vala, how are you.

Vala:   

(00:31) Michael, I’m doing well how are you?

Michael:         

(00:32) I am excellent. So were here with Giorgos Zacharia, who is the Chief Technology Officer who is at Kayak, Giorgos how are you?

Giorgos:          

(00:43) Very well great to be here.

Michael:         

(00:48) And I think everybody knows about Kayak, it’s one of the world’s top travel sites. So Giorgos, please tell us about your background and your professional history.

Giorgos:          

(01:04) So I came to the US in 93 to start IT and ended up doing four degrees there and including computer science. But I’ve spent most of my professional life in start-ups. I started four of them and then joined Kayak at some point. And all of my professional and experience has always had to do with using machine learning to produce valuable information in services and accommodations for end users.

(01:43) Kayak is a prime example of learning from data to bring a better user experience.

Vala:   

(01:50) Well most of our previous guests only have one or two degrees from prestigious Universities so that’s pretty awesome. That’s fantastic and as Michael mentioned many of us know about Kayak, but can you talk a little bit about Kayak and the company and then specifically your role as the Chief Technology Officer at Kayak, what does that mean and what does a typical day for you look like.

Giorgos:          

(02:14) So Kayak’s objective is to build the best app for the user to search and fine the travel product that they’re looking for and then manage that travel even after the booking experience.

(02:31) My role as CTO at Kayak is on technology and product strategy. I spend most of my day working with the product people and the mobile team and the UI development team starting the experiments were about to launch or have launched and monitoring the performance and next on how should we improve the experience next.

Michael:         

(02:59) So you mentioned the term machine learning and that plays an important role in your work and the data science at Kayak. But I’m wondering and it’s a very kind of trendy term these days, but I wonder how many people know what it actually means. So Giorgos, would you give us a little bit of background on machine learning and what does it mean specifically at Kayak?

Giorgos:          

(03:27)So the name of the term comes from the machine, the computer basically learning patterns from data. The way I like to describe it is basically computational statistics. You can have a statistician have hypothesis and test it one at a time, or you can have a computer testing millions of hypothesis in very little time and tell you which pattern has the most predictive value.

(03:54) Practical applications is learning how better to better query our partners to find them the most competitive flight prices or how to better  personalize when we present it to our users. Therefore the preference’s and show them the hotels they are most likely to book hire than three or four pages after the first page of results.

Vala:             

Sure, I mean in your view what is Kayak selling? Are you selling data, are you selling information, user experience, something else or maybe a combination of all of that?

Giorgos:          

(04:37)Well we actually don’t sell anything. So we aggregate the information and try to present in the most usable way. So user experience counts for almost every decision that we make at Kayak, so we try to show our users the most comprehensive results as possible forthere travel search query in the most usable way, whether that’s a mobile device or desktop.

Michael:         

(05:12) So you’re aggregating this data and behind the scenes you’re undertaking various as you said, computational statistics in order to find relationships with this data, but you’re not actually selling anything, so why does the user care?  

Giorgos:          

(05:31) Well the role of Kayak and our mission is to produce competency and accurate data. If we don’t we lose credibility and the user won’t come back. Also we want to present this as fast as possible. For example it makes no sense to have a very complex set of results that might be saving the user a couple of dollars when we know we can get the traveler there with a better flight.

(06:04)So these optimizations matter because you don’t want to make the user wait unnecessarily for a result that they will never book. On the hotel side we don’t want to be querying our partners for every single query and make the user wait for 30- 40 seconds depending on the slowest system that we actually have to query. So we have a predictive cache mechanism,and that anticipates what ouruser are likely to search every day. We pre-empt query all our partners, gather that data locally and send those results as fast as possible from our own cache.

Vala:   

(06:47) Can you give us a sense of maybe a non-technical overview of how the data flows from suppliers  through your systems and then ultimately to the user, and I think our audience would appreciate that whole data flow and the magic behind the scenes.

Giorgos:          

(07:07) Yeah, so we query (countless? 07:12) providers for every query that we see on our system. Typically for flight results we actually prefer to query pricing engines like Hadoop, which now belongs to Google, or Amadeus because that allows us to query the real time inventory of our partners and it synchronizes deep links of the results of the airlines of the travel agencies that will surface results from their websites but we can do that without (entering their websites? 07:46) except airline would actually crash if they saw the daily volumethat Kayak sees. So this allows  usto present reliable and comprehensive results without (entering? 07:58) their systems.

(08:02) On the hotel side as I mentioned earlier, we dothis predictive caching and we still query live multiple numbers of partners if we don’t have that result in the cache, but typically we prime our cache with results continuously so that we can serve them out of the cache when the user shows up.

Michael:         

(08:28) So just briefly and for those of us who are relatively non-technical, can you describe your technical architecture, but go gently on us.

Giorgos:          

(08:41)So for storage of the data and for big data tasks we rely on mostly Hadoop and we also rely on MySQL for other types of tasks and real time serving of hotel data for example. And Kayak is built purely on Java. We use different technologies based on the presentation layer. For our mobile web experience and we transition using Angular (AngularJS?) but that’s a faster development platform and easier to hire for these days.

For the desktop we are using our own home grown architecture that looks like Angular.

Vala:   

(09:35) Can you share with us some primary technical challenges that you are working on or that you face. Whether it’s UI related or mobility related orstorage performance of just all of this data transitioning through multiple nodes. What are some of the biggest challenges that you face as a CTO.

Giorgos:          

(09:35) Yeah, so performance is an ongoing challenge and always to make the system faster. Another challenge we are focused on is patronization. If we know the user prefers non-stop flights for shorter flights from Boston to New York, we need to preselect their one stop two stop flights for the user, especially if they are only modifying the dates. We know the context of that search. If the user has shown preference to particular brands or types of hotels, it’s important that we actually personalize for that user, so personalization in the way that we chose to present the results matters both in mobile and desktop, so that is also an ongoing focus.

Michael:         

(10:46) So what is the link, you talked about the user experience of being the upmost importance and you’ve said separately, that user experience even trumps revenue. Even trumps monetization.

Giorgos:          

(11:07) So any idea we have at Kayak, whether that’s the color or changing the size of the platform, how much information will show in the results page versus when the user clicks to see the different data. All this add a value to the experiment. It could be minor things like I described or it could be a major redesign of the website. Again everything adds a value on the way we experiment. And we measure the user engagement with each part of the website. How fast they can find what they are looking for, and then of course how much money we make. And at the end of the day, we chose the best user experience and giving the same user experience that has the highest monetization for our partners and for us.

Vala:   

(11:59) How often do you change. How often does the algorithm change or highly iterative test, because I’m not sure how you make the decisions but it sounds like it’s a fairly adaptive process

Giorgos:          

(12:14) It is. So in a sense the most convenient way and then given time we run tens of experiments on the website so the user is part of many such of experiments when they come to Kayak. And then once the studies converge and we make a decision and the winning experiment on. We release daily and sometimes as much as four times a day so things can go on and off at any given time.

Michael:         

(12:48) Now Giorgos, what’s the connection between the data you have coming in and the transitions that you are making on that data and the user experience in what the user ultimately takes away.

Giorgos:          

(13:06) Well you could present this data as an Excel spreadsheet or you can  right, or you could present it like Kayak does. So we think that the transformation and the data matters a lot.

(13:19)We also use the data we observe to build new products, and so one example is flight price predictor. We give users a forecast whether the cheapest, where then the cheapest price that they see on our flight result page is likely to go up or down in the next seven days. We also give the user a confidence metric based on how accurate our machine learning has been in making a call in similar patterns.

(13:50)So it’s a decision that the user can actually save money in cases where they have confidence in the prediction, and we observe the users actually engaging with it.

(14:05)Other products that we implement based on the data aggregate is the fares. We observe that in separate cases the user can save money or even travel time issue combined non-aligning airlines like an alliance flight with a flight, so maybe sometimes the way to travel to the West Coast and make it back for dinner is too combined a flight with Virgin America with a JetBlue one. So we sure the users this result that we call Hacker fare, and it requires two separate bookings but it can save the user significant money or time or both.

Michael:         

(14:49) I’ve used those hacker fares

Giorgos:          

(14:54)You’re an advanced user.

Vala:   

(14:55) You’re an advanced – Michael, what was the result, favorable obviously?

Michael:         

(14:59) Yeah, because you save money and you can do it in a way that’s not intuitive, which I guess is Giorgos really the point

Giorgos:          

(15:12) That is the point yes, and we are only able to do this because of the large amount of data that we have.

Vala:   

(15:18) So when you think of all this analytics, predictive and descriptive analytics, prescriptive analytics, what’s the role of predictive analytics in the Kayak product.

Giorgos:          

(15:33)So we use predictive analytics to ensure that we have the most competency and the most accurate results. We actually only remove results after four weeks that we think are no longer available and when the user clicks through to end up on our website we rely on predictive analytics for our personalization and also in the case of price forecasting and hacker fares, we actually do products based on predictive analytics.

Michael:         

(16:11) So one thing that everybody wants to know is how do you save money on booking airfare and hotel. Maybe this is a good time to ask you if any, what are your recommendations for people to save money on buying travel.

Giorgos:          

(16:25)Is different actually, so on flights and I think if you’re flexible and part of (the earlier or 10 day? 16:34) later or you are willing to use in nearby airport and occasionally you can save significant on the future price.

(16:44)We provide tools like price alerts, where if you are flexible on the time of year and want to fly to a specific tourist destination for example, you can have Kayak monitor the prices for you. But my advice, is the same old advice is look early and when you see a price that you are willing to pay, then you should book it, don’t wait.

(17:11)On hotels, because most hotel rates are actually fully refundable if you cancel them. If you see a price that you like, and can go ahead and book it, in the fee cancellation cases, then if it’s cheaper you can then cancel and then rebook.

(17:35) Also a lot of our planners are increasingly working on private rates, and kayak is using these rates for the for the registered users, so that’s another way to save money.

Michael:

(17:52) We have a question from Twitter, Zachary Genes is wondering, what’s the difference with the relationship between the user interface of the software and the overall more broad user experience? Can you link those two, and in your case I think the data.

Giorgos:          

(18:16) Yes, the user interface can be thought of the static expansion of the user experience, and the whole user experience is how you transition from page to page, the animation that you apply when the user plays with the filters in the sorting. So I would say that the user experience is more dynamic for the dimensional aspect rather than the static. And the numbers behind the data science is there to support basically, we capture every single click that happens from Kayak and every single tab that happens in our mobile apps. We see what gets yours in what sequence and optimize it over time to save the user time.

Vala:   

(19:13) I’m assuming there is a head of marketing, and perhaps a CMO and a head of IT, and a CIO at Kayak. As a CTO, which functioned do you work most with and has that changed over time.

Giorgos:          

(19:27) Yeah, the role of the the CIO the chief activator is responsible for that, who rolled out to me. The CMO plays also a key role and so does the CEO and those are the people that I act with the most and discuss new ideas and want to make sure that the user interface is (own brand? 19:53). Our CEO is a very strong person, so we do a lot of remote collaboration through instant messaging and conferencing because there are located in Connecticut. So those are the two that I interact with the most other than my team.

Michael:         

(20:19) We have another from Twitter, Arsalan Khan and asks what is the future of data analytics gone beyond user experience. Where is this all going.

Giorgos:          

(20:37)Yeah, well at Kayak is mostly focused on user experience and ensuring comprehensiveness. The future of the product of Kayak is more mobile and more international.

Michael:         

(20:50) So tell us about mobile, mobile plays a big role at Kayak

Giorgos:          

(20:54) It does. More than half of our users are on mobile devices. If you look at our mobile usage, more than half of the searches for flights are for one way trips and more than half of the searches on hotels are for tonight stay’s or tomorrow night stays. So our users are using it as a way to sort a travel problem actually. They are stranded and they need to get out of there, or they need a car all hotel right away.

(21:30) If you ignore the fairs in the next two days, the (hotels? 21:35) are actually identical to desktop (so they are using that medium and expect the same as a desktop replacement? 21:39). So for expedition searches far out into the future and in very similar ways that they would do on a desktop.

(21:52)  So we will strive to make sure that in that small screen we provide as much functionality as possible. Now in reality, well increasingly the user might do some other expedition searches while watching TV on their mobile device and they come back later to book it on the desktop. So you will see Kayak being more supportive on these mobile device used cases.

Vala:   

(22:17) So from search action, I’m guessing on mobile that maybe you have an order of magnitude that is faster than on a desktop. Is that a fair statement?

Giorgos:          

(22:25)It depends on the number of (unclear 22:30)

Vala:   

(22:34) Okay, so I suspect over time you know, your smart phone, tablet that is going to be the first screen when it comes to in terms of Kayak and you said today what it’s 50% of search comes through mobile?

Giorgos:          

(22:44) 50% of the users yes and maybe a bit less than that. Already, our design is mobile first, so we have the mobile design team coming up with the next iteration of the design of Kayak and then the desktop expands for the whole of Kayak. We make sure that increasingly our desktop experience is optimized, so mobile is truly driving the identity of Kayak on the devices.

Michael:         

(23:21) So you develop personas for your users as well, so for example one of your personas is the stranded traveler as you were describing earlier

Giorgos:          

(23:28) Yeah, so that’s where personalization comes into play. The results you want to show to the stranded traveler may be different than the aspirational searches and that is ongoing work as I said, speed and persona personalization never ends, so those are the two key priorities that we’re always focused on.

Michael:         

(23:57) Well as an occasional stranded traveler myself, sometimes due to my own fault, I booked a flight to the wrong city, which I can tell you is not a pleasant thing. Not at all.

Vala:   

(24:07) Was that a speaking engagement Michael?

Michael:         

(24:08) Yeah let’s not go there. But the first thing I did was when I got off the flight was that I opened up the Kayak app on my iPad because I figured that’s going to be the fastest way for me to figure out how the hell do I get out of this city and get to where I need to go.

Giorgos:          

(24:28) Yeah, that’s a common story we hear and we even hear about pilots using Kayak to find out the schedules of flights and I’m glad we were of service there.

Michael:         

(24:39) Well you know I’d definitely appreciated it at the time. But we’ve been talking about the data part, now let’s talk about your business and your culture. You’re from a size perspective and time in business and you not really a start-up, but you have the self-image of being a start-up.

Giorgos:          

(25:01)We do, and I think that’s important for retaining the talent but also remaining productive. If you looked at our team, about 70% of the team is actually technical staff. Engineers, designers, QA, data science etc., and maintain small teams in a very flat organisation. We are very light on process and we have meeting phobia, so we rely on having very intelligent and motivated people who can work together very efficiently in small teams. I believe that has been key in Kayak’s execution.

Vala:   

(25:49) Can you talk a little bit about meeting phobia, because after all it can be about more than 20 minutes, you have to stand up – how do you address meeting phobia.

Giorgos:          

(26:00) Again, it is addressing a very organic way if somebody sets a meeting invitation that has too many people, you would get rejections because the number of people is too large. I don’t think I have been in any meetings at higher level that have been too long, and people organically say that this is too long a meeting and it’s not practical and we cut it off. Also in our offices we have stand-up meeting rooms to maintain that the meetings remain short and practical.

Vala:   

(26:37) That’s great advice for all companies.

Michael:         

(26:40) Now what about hiring, so your hiring lots of data scientists, your hiring user experience people and you don’t just simply want them showing up and knocking on your door or am I wrong about that.

Giorgos:          

(26:56) No we don’t we enjoy hiring people out of our network. We sometimes higher recruiters, but our most successful hires came through a referral. Even the ones that we hire through recruiters in terms that if they end up connected a prior Kayak employee. And that’s very important, because we are light on process. We want people who can work with each other and enjoy working with each other, and our hiring process actually always tries to enforce that after a candidate goes through a number of interviews and we get together and any of the interviewers has a bit of power. So you have to get the thumbs up from every single person that you have met to be hired at Kayak.

(27:53) Also on the product side and if you market on social network of the Kayak employees, you will notice that we basically have four or five very strong clusters of people who knew each other prior to, or somebody who knew them prior to coming to Kayak.

(28:16) So IQ is important and filtered before we invite somebody for – and Kayaks success is important is filtered before we bring somebody to Kayak. But being compatible from a culturist perspective is (unclear 28:31) so we need people who we can work with efficiently.

Vala:   

(20:37) So you hire for attitude and trained for aptitude if that makes sense. And does that allow you to scale when you’re looking for data scientists and UI experts from what we read today, that there seems to be a shortage of highly skilled precision talented employees. So when you’re networking your referrals helps you find the right candidate.

Giorgos:          

(29:08)Exactly it’s about finding the right candidates. We’ve got thousands of positions, but we are not going to hire to fill up those positions. We want to hire the right people at Kayak.

Michael:         

(29:17) So how do you maintain a culture that combines the very very highly technical discipline of data science with sensitivity and awareness of user needs that is required to build the system that is very responsive to the user experience. How do you find people that can marry those qualities together?

Giorgos:          

(29:47)That’s mostly personality, and that’s what we try to find out when we interview somebody at Kayak. We don’t want people who are in love with technical frameworks and they have no sensitivity about the end product that what we are delivering to the users. The way we value it and the way that every IT architecture change is they would be experiments. If you change that structure in a way that affects the user experience, that experiment fails and we throw it out.

(30:17) Also when it comes to team productivity, I have had persons that I’ve let go that were very intelligent people who were not the right culture or framework with our team, so we iterate on the team composition like we do on the product.

Vala:   

(30:32) That’s incredible, so obviously a customer first accountable culture.

Giorgos:          

(30:41) I think it’s team first and then customer.

Vala:   

(30:46) That’s fantastic. How did you look at traditional resumes versus someone’s digital footprint or social network when you are going through the recruitment process?

Giorgos:          

(30:59) In most cases we first get the traditional resume or LinkedIn profile and we like to look at (data? 31:07) or there portfolio if it’s a designer. We like to see Kayak working basically on that code and we like to see what that person has done before.

Michael:         

(31:20)So you’re obviously a very innovative company and you are constantly doing in essentially innovation experiments all the time. So tell us how you maintain the pace of innovation and tell is the challenges of doing so and what you’re doing there.

Giorgos:          

(31:38) I’ve never been happy with your execution speak, I think that ensures that you keep looking for inefficient processes that you can remove, or that you can remove projects that you shouldn’t be working on, because they don’t move either. And I think that has to be ingrained across the culture of the team.

Michael:         

(32:01) Do you have a formal innovation process in place.

Giorgos:          

(32:07)The most formal aspect that we have is AB experiments. We also hold a half week were we encourage our team to come up with new ideas, but the kind of ideas that we would release on production. We also have ad hoc week or half month. If any of our technical staff comes up with a great idea that we would like to implement, depending on their execution history, will give them that ability to work on that project on their own.

(32:44)We also slice out teams to work on moving shop projects that are interesting and tangible ideas, so we give them that undistracted from daily responsibilities and some of those projects work and some don’t. The ones that do end up getting released on Kayak.

(33:08)Our mobile product was actually sliced out to teams that we protected from our CEO long enough to reach the test.

Vala:   

(33:20) I know how that feels. So you know, we’ve had in the past several start-up CEOs and start-up founders on our show. What advice would you give to start-up, CEOs who want to come and pitch their capabilities to you as the CTO in terms of ensuring that they could survive the first 10 minutes of the meeting?

Giorgos:          

(33:44) Yeah, we get a lot of young start-ups pitching new products etc. it has to be interesting to Kayak, it has to be solving a mass consumer need if it is a highly aspirational product or if it is a very niche travel related product then that would not look at it very seriously. The mass consumer is key for us.

Michael:         

(34:14) How do you optimize from a marketing standpoint, how do you optimize your own site to maximize traffic, to maximize clicks and maximize the revenue as well.

Giorgos:          

(34:28) So that is the brand marketing aspect of things, but we also do some online marketing, which is performance marketing and I think that’s that’s a mass problem as far as we’re concerned. We want to bring users who have a good user experience that it would help to kayak in the long run it is a positive (unclear (34:47) for us.

Vala:   

(34:49) What are some of the things you do to maintain this start-up like entrepreneurial and obviously customer user experience obsessed culture at Kayak.

Giorgos:          

(35:02)With some of the things we do is that we expose every employee to customer feedback when a user writes to Kayak, that will be randomly routed to a Kayak employee. That has interesting benefits, if an engineer keeps seeing the same problem and tired of asking the same questions, they just solve the problem. So the constant diet direct contact with the customer helps you learn what you are doing wrong.

(35:32)Customer monitoring of usage of the website and pruning all the features that are not valuable, basically ensures that there is that optimization and it keeps you focused on the customer experience innovation.

Michael:         

(35:53) And your role is CTO, and it seems to a large extent that you also have to play the Guardian of the culture.

Giorgos:          

(36:06) Yes, also we like to joke that we maintain separation of the church and state of Kayak. So the product team is responsible for the user experience, and the commercial team is responsible for commerce and monetization, and there is constant push and pull and the one with the strongest opinion ends up winning.

(36:31)The reality is not clean separation. Some of us on the product side also care about monetization, and in our commercial cares about user experience as well, and I think that has worked well for Kayak.

Vala:   

(36:46) So you said that the strongest opinion wins, but clearly you are getting principle and core values because you had mentioned user experience almost always trumps other factors, so at least there is a certain element in the DNA of all of the employees at Kayak to ensure that you are delighting your customers to the best of your ability. Is that fair?

Giorgos:          

(37:09) Yeah, that is very fair and that is from the CEO down to the single employee.

Vala:   

(37:14) So how do you capture feedback, so what are the mechanisms and how much feedback do get on a daily basis. You don’t need to be specific, but it sounds and I’m assuming it’s a ton.

Giorgos:          

(37:27) Yeah, it’s a ton we get a lot of emails and that is mostly how we capture feedback, but we also monitor Twitter and other social networks and increasingly feedback also comes back from social networks and this is the active user feedback capturing. But there is also the passive one where we observe our users behaviour on the website and we learned from that and optimize from there.

Michael:         

(38:03) Now, the whole concept of data and predictive analytics and machine learning is one of the big trends today and everybody is talking about it. I wonder how many people are doing it in an effective way, but can you offer advice to start-ups or to larger companies who are trying to or at least get their feet wet and use these techniques in a meaningful way. Based on your experience and what you’ve learned, what are some of the challenges and what are some of the things that they should look out for?

Giorgos:          

(38:39) Yeah, first of all I cannot think of any viable mobile solution behaviour on big data and customer optimization. The big challenge is to capture the right data and make sure you actually measure it correctly. So we have spent a lot of time at Kayak optimizing our platform to actually do the right types of segmentation and on reliable statistic data. Location really we might experiment that doesn’t capture the right type of data, so it requires constant attention and diligence in asking the right questions when you see data that doesn’t make any sense.

Vala:   

(39:34) I’m curious and maybe you can give some advice to other CTO’s, and we have had numerous CIOs and other technology executives as previous guests. What are some of the conferences or what do you read, and how do you as the person who is supposed to you know who is responsible for championing the vision and product strategy for Kayak. What do you read, where do you go and who do you speak with to stay not ahead of technology at Kayak what are some of your sources?

Giorgos:          

(40:07) Yeah, as I said earlier that we have meeting phobias, so we are not big fans for attending conferences. Personally I read (sounds like machine learning 40:20) and also user experience. But mostly we hear from users by mobile devices ourselves, we are travelers ourselves and we use our competitors’ products to maintain that we have the most comprehensive results. We also use a a lot of other mobile applications in our daily lives and user product experience, which makes sense and we discuss in help we can use this in Kayak.

Michael:         

(40:57) So we are just about running out of time and I’m always interested in what are the kind of biggest challenges that you think about. You mentioned about having the right type of data and taking the right type of measurements. Are there any other things that keep you awake at night?

Giorgos:          

(41:25)It’s the position of the market device work to make sure that the device works as users start their searches on there or mobile and then they end up booking on a desktop or the other way around. We need to make sure that we support that used case as well as possible. I don’t think we have cracked it well yet and I don’t think anybody else has, so I think it is a graceful position to a market device work.

Vala:   

(41:55) So when Apple introduced the new watch and had voice activated Uber service as a demonstration, whether it is Google glass in the future or a watch, how do you see wearable technology where folks will want to perhaps use voice technology to search and interact with Kayak? Is that something that you see as potential growth area or usage area?

Giorgos:          

(42:21)We actually experimented with voice search, but I think it was too early when we did. We will take a wait and see and we’ll go where the users are.

Vala:   

(42:34) Makes sense.

Michael:         

(42:34) Fantastic, this has been a very interesting discussion. We’ve been talking with Giorgos Zacharia, who is the Chief Technology Officer at Kayak and we’ve been talking about data, analytics, and machine learning. Giorgos, thank you so much for taking the time.

Giorgos:          

(42:56)Thank you for inviting me.

Vala:   

(42:58) Thank you very much, we’ve learnt a lot and look forward to engaging with you in the future.

Michael:         

(43:04) Yes it’s been great. I am Michael Krigsman and this has been show number 102 of CXOTalk with my co-host Vala Afshar and I hope everybody has a great week and we will see you again next time.

Companies mentioned in this week’s show

Kayak                                      www.kayak.com

Google                                    www.google.com

Amadeus                                www.amadeus.net

Virgin America                       www.virginamerica.com

JetBlue                                     www.jetblue.com

LinkedIn                                  www.linkedin.com

Twitter                                     www.twitter.com

Uber                                         www.uber.com

Apple                                        www.apple.com

Published Date: Mar 13, 2015

Author: Michael Krigsman

Episode ID: 102