This month, Skift announced an AI chatbot called Ask Skift that readers can ping with questions. Answers come from Skift’s 11-year archive of stories and reports, as well as public financial documents from US travel companies. Within its first 48 hours, the bot received between 3,000 and 4,000 questions, Skift’s founder and CEO, Rafat Ali, told Marketing Brew.
We played around with the bot, which was built on OpenAI’s ChatGPT, asking where we should go on vacation this February. It couldn’t answer. But when we asked who Marriott’s CEO is, it replied with John E. Geller, who’s CEO of the company’s timeshare business, Marriott Vacations Worldwide. Close, but Google provided a more accurate answer (Anthony Capuano). When we asked how much Marriott spends on advertising, it pulled figures from 2020 and 2015.
Like most generative AI, it’s advisable to fact-check. But the travel industry site’s investment in AI chat is a zig when it seems like every other publisher is trying to squeeze generative AI for content.
Marketing Brew spoke with Ali about Ask Skift and the company’s AI ambitions.
This interview has been lightly edited and condensed for clarity.
Was this something subscribers wanted?
The way I look at it for us is, internally, how can we use these tools to make us more efficient? Externally, what can we launch that takes advantage of what we’re best at, which is expertise? What do we do with this body of knowledge that we’ve built over the last 11 years of expertise covering the business of travel in deeper ways?…[Say] you’re a hotel revenue manager at Marriott. You have a quick query that you want to figure out—what’s the market share of the top three hotel revenue management software companies?—you ask a bot. Turns out we have that answer buried in five different stories or reports that we’ve done. [It] just unlocks the archives.
Historically, publishers have done a pretty poor job at internal search engines to access their archives.
Right. If you generally think search is broken at a Google level, imagine it’s 10x worse at the publisher level.
How many queries do you expect to get a month?
I have no idea, I have no fucking idea. In the last 48 hours [since launching] we’ve had 3,000–4,000 queries. That’s a lot for a B2B publication. For now, it’s just a novelty—people just checking out what it is. It’s three queries a month free, which basically mirrors our subscription, which is three stories a month free, and then you have to pay for Skift Pro…This is definitely aimed at our paid subscribers…Anybody who has a paid relationship with Skift, you get it as part of your subscription. At some point, we’ll figure out if we want to upsell or [if] we want to increase the prices and say this is a value-add. Or, do we do a standalone thing for this, which I don’t know if that makes sense…And retention is a big thing, obviously. If you’re using us a lot more, you’ll stay…Not only do we give the answer, we also give the three top articles we sourced to get the answer. The reason we’re showing those is because, hopefully, it increases page views per user.
Get marketing news you'll actually want to read
Marketing Brew informs marketing pros of the latest on brand strategy, social media, and ad tech via our weekday newsletter, virtual events, marketing conferences, and digital guides.
What’s the breakdown of the company’s revenue?
It’s 30% subscriptions, 35% advertising, which for us is all branded content—we don’t do banners and box-type stuff, it’s all custom work that we create for companies—and 35% events.
Do you see Skift reporters using these queries to mine stories?
100%, to research their stories…It definitely should be and will be an internal tool as much as an external tool.
Are you concerned about AI hallucinations and falsehoods?
We have to be careful what we train it on…The ironic part is if you train it on too much, you will have hallucinations, or wrong answers, or answers that are not current. You have to fine-tune the algorithm…You give weightage to different things, so the weight is given to recency first…Our research reports have a higher weightage than news stories…In the last 24 hours since launch, it became apparent that we need an internal content director that finds new sources for this to train on, figures out what not to train it on, and figures out potentially proprietary sources of content or data to train it on.
What’s the cost associated with training the algorithm?
The cost, in general, to launch this product that you’re seeing, is minimal. It’s two developers that were focused on it over the last eight weeks…From a staffing perspective, this is the beauty. This is the incredible part of the tech, which is you just don’t need that many people. You also don’t need AI scientists or data scientists; all you need is developers.
Is AI writing any stories on Skift yet? Is that something you’ve considered?
No, we’re not going to write stories with it. But this is an example. Yesterday, there was a prominent online travel executive who was a pioneer who unexpectedly passed away. We did a reported obituary of him; we were the first ones to do it. He was important, he was somebody who was quite well-known, but not so much that everybody would know the details off the top of their head. I was helping the edit team do some background research, and I put into ChatGPT “create a bio for Hugo Burge,” and it came up with six or seven actually good paragraphs. I sent it to the edit team to say “This is raw material, fact-check it, use anything you think makes sense out of it.”...It’s a raw material that helps us today. Are we doing this at scale? No.
How will you measure success for the AI chatbot?
Intuitively, it’s engagement, retention, new users, [and] new paid subscribers. Those are the two metrics any subscription-driven company would want.