“The bulk of linear is still bought at a corporate level, so until those buying habits change we’re not going to see too much movement there,” Khan adds in this interview at Beet Retreat 2018.

WideOrbit has long provided the foundational piping for local TV broadcast transactions. After several acquisitions of digital assets, its programmatic solution now consists of four “flavors,” according to Khan. They are programmatic direct and private marketplace, which are “more akin to your traditional buying RFP’s” for guaranteed inventory that can be bought one year out.

“Then we have open marketplace and direct response, more akin to your open RTB protocol” in which spots can be purchased in real time anywhere from two to seven days out.

Meanwhile, on the demand side, WideOrbit engages with brands directly, “we do work with the DSPs, and we service our core SSP client needs in that we provide the channel for direct sold.”

Khan mentions WideOrbit’s 2017 partnership with Google, in which WideOrbit integrated with DoubleClick to allow ad buyers to buy broadcast TV spots from the same interface they use to secure digital display and video campaigns.

“We continue to see increased volume through that channel. We work with a number of DSP’s as well, and again we continue to see increased volumes coming through those demand channels,” says Khan.

To advance the convergence of digital and linear media, WideOrbit has developed different tool sets. “We are helping with the data and the targeting through our programmatic platforms. “Brands don’t necessarily always want to target a specific audience. They like the shotgun approach of linear. It helps with their brand building, and obviously it’s the safety that goes along with that as well.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

AT&T used a local meteorologist/social influencer to help inform and people prepare for the hurricane, while providing journalists with free WiFi data cards afterward and using satellite television programming time to inform children sidelined by school shutdowns. P&G provided free mobile laundry services so that people had clean clothes to wear. Telemundo took to using a live Facebook feed to communicate news developments while hosting gatherings of friends and competitors alike in the industry.

Beet.TV’s efforts to galvanize support for Puerto Rico relief efforts continue on February 5 at Xandr/AT&T in Manhattan. Accompanied by advertising and media executives, the program will explore fund-raising and other charitable initiatives to further support the Boys & Girls Clubs of Puerto Rico.

“The media landscape was basically starting over again,” Andres Claudio, MD of Hearts & Science/Omnicom, Puerto Rico, said during a Beet Retreat session titled Puerto Rico Media Transformed by Hurricane Maria and moderated by former GroupM executive Phil Cowdell. “Imagine in New York City if you have no power, no signals, no cell phones, nothing. At least you have another state around you.”

While brands have a responsibility to understand consumer needs, “we have a social responsibility” as well, said Melissa Burgos, Marketing Director, USVI/Puerto Rico, AT&T Mobility. In the case of Maria, that responsibility started with the mobile service company helping people prepare for what lay ahead. AT&T leveraged the effort by using a local meteorologist who had a substantial social media following.

“What we did not expect, besides the fact of what happened after the hurricane, was that influencer became the top trending person within all possible influencers,” said Burgos.

“Basically we were in the dark ages” in terms of the ability to communicate through everyday media, with the exception of AM radio, said Freddie Hernandez, Site Leader, P&G Puerto Rico. When the company saw images of people washing their clothes in rivers and other waterways, P&G knew it had to do something.

“There’s a lot of health issues related to using non-purified water to clean your clothes,” said Hernandez. And while P&G had learned from such U.S. natural disasters as hurricanes Katrina and Sandy, it had no logistical way to bring its Tide Loads of Hope community service from the mainland. “We decided to create our own asset” in the form of a Coach bus transformed into a laundromat with 12 combo units of washing and drying machines that assisted more than 2,000 families.

“It’s incredible the reaction of the people,” said Hernandez. “You take for granted what you have. People were just crying when they would see the bus getting to a town and people would be wearing clean clothes for the first time in about three weeks.”

Telemundo had been broadcasting “all the way up to the storm” but was forced to shut down early on the first morning that Maria made landfall, according to Jose Cancela, President of Telemundo/NBCU Puerto Rico. “When we cranked back up we were really broadcasting to the States because there was no power here. Whatever cell phone probably you had you probably used it up,” said Cancela. Reporting news live via Facebook was “for four days how we were broadcasting. Radio became the media of the day for a number of months.”

From the onset of Maria, advertising and media competitors pulled together, initially using Telemundo as a base for meetings. Meanwhile, Cancela credited Nielsen for doing “a great job in bringing their panel back on track. Without that measurement, our currency was zero and it wasn’t until May of this year that they were finally able to bring it on track. But during the whole process, they really kept the broadcast community and the agency community very much informed in getting their panel back up.”

One exception to media outages was satellite TV for those fortunate to have a source of electricity, said Burgos. Since all schools were either shut down, destroyed or being used for community services, “we had children at home doing nothing, so there was a need to educate or entertain those children.”

So AT&T organized a tour of education professionals under the auspices of DirecTV and used programming time to provide useful content to kids in 10 different towns. “They were dying for information,” Burgos said.

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

Moderated by consultant Matt Prohaska, the discussion touched on the eternal reality of supply versus demand in making sales decisions, along with the prospects for the OpenAP audience targeting consortium.

“We still have bifurcation in linear programmatic and digital, so that’s always something that we have to navigate,” said FOX’s Noah Levine. “We’ve seen a maturity begin to develop in the linear programmatic TV space, which is quite nice.”

From the sell-side, programmatic is “primarily about automating the buy and being able to empower the agency, the buyer, to do more. That’s a good thing for us as sellers,” Levine added.

“The fact that we still need to look at linear programmatic as kind of a separate beast is something we should all start to want to reconcile and maybe have some degree of concern about,” said Brett Hurwitz of OATH, the Verizon unit. And if all TV inventory becomes available on an IP-based delivery platform, a major concern will be getting “enough of a premium on the highly desirable target folks to make up for the fact that some of your other impressions are going to probably be going at a much lower price.”

The issue of brand safety still holds sway and stands in the way of 100% data-driven decisions, according to Mike Law of Dentsu Aegis Network. “We need to find the right balance of that, because some brands hold that way too close to them and some buyers hold that way too close to them,” said Law. “They fear that if I don’t say something then this computer will do my job for me or somebody will do it for me.”

Sellers have obvious concerns about total automation when optimizing their inventory across the multitude of buyers. “In linear, there’s a lot of pressure on the inventory. There’s a huge amount of demand,” said Levine.

“It’s true, the private marketplaces are really the path forward for the most part when it comes to especially linear television inventory,” said Hurwitz.

Given some marketers’ desire to cherry pick ad units versus having the ability to transact via automation, “There’s a very healthy tension between those two models that we’re seeing in the marketplace,” Levine noted. And while being able to leverage programmatic technology to access inventory and re-optimize plans “is a very desirable state for us to reach in the future,” it’s tough to do for sellers that don’t have lots of unsold inventory. “That’s one of the challenges.”

Asked by Prohaska about the prospects of the OpenAP targeting consortium launched by FOX, Viacom and Turner, Law said its premise “remains really strong and positive.” The missing piece to him is being able to transact collectively across all members.

And OpenAP is in a crowded space.

“There’s OpenAP and then I’ve got actually all the partners included in OpenAP trying to sell me their own platform, plus fifteen other networks trying to sell me their platform as well,” said Law. “And then I’ve got Simulmedia calling me, I’ve got Adobe calling me, I’ve got Videology calling me.”

OATH experienced its own version of too many choices upon the merger of AOL and Yahoo, Hurwitz recalled, given the existence of “I’m going to say eight DSP’s between the two companies” plus a number of DSP’s. OATH decided to sunset its linear TV programmatic platform, which Hurwitz called “a fantastic product,” because “we felt was going to serve the industry for some period of time and then perhaps not be the tool the industry needed.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“We kind of fill the gap that the education system leaves in Puerto Rico,” Ramos explained in a one-on-one discussion with Phil Cowdell, formerly Global President of Client Services at GroupM.

In Puerto Rico, nine out of 10 schools “are failing schools” based on standardized tests, according to Ramos. Add to this some “crazy stats” cited by Cowdell: Maria damaged 82% of the island’s homes, 42% of the population had daily food shortages and 44% of children have a registered mental health impact from the storm.

“That’s pretty cataclysmic and we’re talking about this is part of the United States of America,” said Cowdell.

The bottom line education-wise was that children in Puerto Rico missed a half a year of schooling in a system already burdened by systemic failure.

“In Puerto Rico, we tend to be short minded, so we forget really quick, and we will say that we reopened schools six months after and that we passed our students. We pass them,” said Ramos. “How do you insert those kids in the formal economy later on, ten years from now?”

In the aftermath of Maria, her organization reframed its services to include things like yoga and mindfulness “so we can try to help the country doing the job they should have done with the storm.”

Having been involved early on in emergency relief efforts, Cowdell recalled that just over a month after Maria, “the far end of the island hadn’t seen milk for six weeks.” At a Costco store, “there were people there buying flat screen TV’s because they live on Ashford Avenue in the downtown part of San Juan.”

His point was that when the storm first made land, “it was a very democratic storm, it impacted everybody. It became very economic the week, the month and then the period after it. And what we see now is an economic impact.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“We view it as a deterministic match between two first-party data sets,” Arrix, who is SVP of DISH Media, said of the addressable designation.

“I think that is really important because there’s a lot of faux addressable out in the marketplace that is disguised as data-driven linear, and I think it’s really important for the marketplace to understand what is true addressable and what is not.”

For their part, DISH and sister entity Sling TV aren’t looking to “create automated tools or platforms per se” but rather see things from a predominantly supply side partner point of view, according to Arrix.

“One thing that we’re trying to do to make it easier for the market is combine our full addressable footprint of DISH with Sling,” Arrix said. “We’re not trying to break it apart and say this is linear addressable and this is OTT addressable.”

One year ago this month, DISH partnered with Comscore to launch a cross platform/screen solution encompassing inventory on both DISH and Sling.

Shimmel, formerly of Turner Broadcasting and now President of Janus Strategy and Insights, assumed that over time Sling will become more of the companies’ distribution footprint than DISH “due to natural things going on with consumers. Is that a good thing for your ad capabilities?”

Arrix drew distinction between DISH trying to maintain its pay-TV subscriber base and ongoing set-top box migration, since about 70% of DISH’s boxes are addressable. “While the overall DISH subscriber base, TBD what happens there, the addressable footprint of DISH will continue to grow,” said Arrix.

Asked about Comscore’s role, Arrix said the company provides third-party measurement of addressable impressions that are unified in a single report. “It’s not this set of impressions because they’re coming from a set-top box is measured this way and this set of impressions I measured this way. They’re connected in with the technology that’s actually driving addressable on our side, which is Invidi, and they’re able to deliver one pool of addressable impressions.”

As for third-party measurement in general, Arrix said, “We advocate for and partner with any third-party measurement company and we believe that it really is up to the demand side, marketers and agencies, to determine which measurement they want to go with.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

The short summary of their back and forth would be that from a cross-platform perspective, TV is still “a mess” and that despite much talk about industry collaboration among competitors, reality is bound to intrude when companies like Comcast and Disney wage their inevitable marketplace battles.

Gerber has been on the sell- and buy-side and is currently President of Global Media Investment for GroupM’s Essence agency. Clark is EVP of Advanced Advertising at Comcast & GM of FreeWheel.

In trying “to make supply make sense,” Comcast has scooped up no fewer than five companies—from FreeWheel to STRATA, under the FreeWheel brand—with the overarching goal of unifying linear and digital by removing a lot of the fragmentation and friction among the tv infrastructure, working on addressable technology, decisioning, and making life easier for marketers at the end of the day. The competitive backdrop is the ease of use of such platforms as Amazon, Facebook and Google.

“Whether you’re a programmer or you’re a marketer, you cannot run one campaign across TV. You cannot have it optimized in real time, you cannot get a report. You have to go hire an army of people to do that,” Clark observed.

This is why “television is not a platform…it is hundreds of sales teams, different ways of selling depending whether you’re local or national, measurement’s all over the place.”

As for Amazon, Facebook and Google, “Yes they’re walled gardens but they are unified tech and data platforms and they can bring sort of a seamless solution to marketers there’s no question about that. TV is not that,” said Clark. “Programmers are unified in wanting one campaign across linear and digital environments, which FreeWheel serves against all–including STB VOD–so that programmers can have everything unified.”

Never one to beat around the bush, Gerber put aside digital/linear unity for the more timeless question of competitive media selling, particularly as it pertains to national TV avails. “The fact of the matter is, most inventory still trades in television in a traditional unit linear model and it’s controlled by independent sellers,” said Gerber. “I care about the national avails that the national networks control because that’s where the bulk of viewing occurs. I need that brought together with the local avails, with the digital avails, everything together on one platform like you said with Google and Facebook. I don’t see that happening.”

Clark recalled when TiVo launched in 1995 and some people predicted “this is the end of television, it’s all going to change.” From his vantage point close to technology, he likened the state of affairs as “driving towards a mountain on the horizon, it doesn’t look like you’re ever getting close to it and then one day it’s right in front of you. There are massive investments right now going into the technology needed to clean this up.”

Gerber: “Who’s your competitor? Are you guys all going to work together to help us figure this out? Are you all going to compete with each other?”

This is where reality intruded. Just one day before, it was announced that Disney had chosen Google over FreeWheel to handle its digital advertising, as Advertising Age reports.

“They make the decisions that are in their best interests of course,” said Clark. “It’s undeniable that Google now has a massive beachhead in the industry. There’s no question about it.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

Experian helps advertisers determine the data segments that will perform best and use those segments more seamlessly. In this interview at the recent Beet Retreat 2018, Danaher talks about the company’s Audience Engine platform and heightened concerns about consumer data privacy.

The company launched Audience Platform in 2016 to facilitate cross-channel audience targeting and closed-loop analytics. Citing one function, Danaher says the platform takes in data from MVPD’s and “can quickly, within seconds, determine the size of the audience that the advertiser wants. That is something that is removing a pain point.

“It’s often been very cumbersome to create segments. And there’s still some difficulty there, but this platform helps to speed up the precise of sizing up the segments so that then the media plans can be put into place a little quicker,” he adds.

Both the buy- and sell-side have similar concerns with regard to advanced audience targeting.

“The buy-side is trying to solve their advertisers’ desire for an interesting segment, one that is effective for them,” he says. Experian provides its own data sets “and we also have a team that helps them craft it together. A bit of a service model.”

On the sell-side, “they also need help guiding their clients and just understanding data, and also quickly matching first-party data as well as using that third-party data.”

Consumer data privacy was one topic of discussion at Beet Retreat 2018. Danaher says the company understands the concern all too well.

“That is actually in line with what we’re all about at Experian. As a credit bureau, we have all the rights to use it as a credit bureau, but also in our marketing data that is all opt-in data. It is not behavioral data or taken from online sources inappropriately.”

Looking forward, Danaher says Experian is very focused on identity, linkage and advanced data and measurement. Using data that’s effective and then measuring that on the back end “will prove the value of the data and prove that you did it right. That’s the advantage of targeted TV is that you can see it turns out and very clearly the cause and effect.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

NinthDecimal specializes in “physical world data” that, among other things, help advertisers understand the impact “that television ads had on where you’re going, meaning did you shop more because you saw an ad” by tying mobile devices to households,” Kilmer says.

In this interview at last month’s Beet Retreat 2018, Kilmer talks about the quest to properly attribute to advertising media like TV business results that are being credited to other media and how marketer KPI’s can change given campaign results.

NinthDecimal can tell whether someone visited a quick-serve restaurant after viewing ads not just on TV but in digital, mobile and out-of-home media.

One of “the more interesting tactics we’re working through in television” is helping to determine whether the premium cost of a commercial pod takeover drove enough business outcomes to justify the premium “or is it more effective to just buy thirty-second spots or fifteen-second spots,” Kilmer says.

For theatres, primetime TV spots on Thursday night are a mainstay of movie premieres. But does a film also need to be advertised on the weekend after the opening? “Or can they supplement that advertising with different forms of marketing to drive consumers into theatres?”

Apportioning the right level of credit for advertising “is a hot topic for me personally,” says Kilmer.

“We measured a campaign last year that reached ninety-five percent of U.S. households with their linear TV buy. And we were measuring their digital ad effectiveness as well. Which simply raised the question, is TV doing a lot of the work that digital media’s taking credit for?”

While marketing and media modeling techniques abound, Kilmer would like to see an analysis within a consistent environment “to be able to truly understand the impact that mass awareness and the upper-funnel activity does have on the digital-based media that we often drive life from that is getting a lot of value based on other advertising.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

Data Plus Math has also partnered with Inscape, IRI, LiveRamp, Verance and others in its quest to supplement expensive post-campaign analyses with in-flight learnings that can be used to optimize campaigns on the fly.

When the company was launched in 2016, with investors that include Comcast Ventures and Greycroft Partners, it saw an opportunity to bring attribution to TV.

“Our main idea was that TV advertising works and the data would tell a good story if someone would go in and actually analyze it ad show that television is driving real world outcomes for marketers,” CEO & Founder John Hoctor says in this interview at the recent Beet Retreat 2018.

Data Plus Math started by building a signal graph that uses advanced data science and automated machine learning techniques to map actual ad exposure data to consumer behaviors.

“There’s been lift reporting and consultants and expensive studies. These studies usually take weeks to do. You usually get them weeks after a campaign is over. We wanted to have fast, low-cost attribution,” Hoctor says.

Going beyond consumer panels, the company prefers to work with “more of a footprint” of TV viewing data from set-top boxes, smart TV’s and streaming. “We have our own impression tracker that gets trafficked along with the creatives and can bring back exposure data from OTT or full-episode players,” Hoctor adds.

“It really helps TV get the full credit. If an advertiser’s trying to drive a certain attitude toward their brand or show that their brand is a fun brand, for instance, Millward Brown’s panel is really good at doing that.”

Describing A&E Networks as “great, forward thinking partners,” Hoctor says Data Plus Math is helping A&E to use attribution “not just as a report card. Attribution sometimes can be seen as you run the campaign and attribution will tell you if it worked.

“What really gets me excited is closing that loop and being able to not just measure whether something worked after it’s over, but measuring it in flight and providing back levers to help optimize for it while the campaign is still in flight.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

In fact, it’s “nirvana in a sense,” O’Grady adds in this interview at the recent Beet Retreat 2018. He was interviewed by TV research veteran Howard Shimmel for Beet.TV

“It’s where all marketers are heading. They want to know for every dollar they spent what’s the return on the investment.”

Not that it’s quick or easy, like perusing overnight television ratings. “So we’re lucky to be in that business, we’re lucky to have all these assets, but a lot of it is still manual. A lot of it is putting the pieces and parts together.”

Things are headed beyond just activation targeting and measurement. “It always has to start with planning. So we call that purchase-driven planning,” O’Grady adds.

Asked for his view of consumer data mishaps in general and the recent Marriott breach of its Starwood guest reservation database, which exposed the personal information of up to 500 million people, he points to Catalina’s “flawless track record” on the use and protection of such data.

“Data protection was always on our mind before privacy was branded on our forehead,” he says. “It’s actually the retailers that have the permission base to use the loyalty card data with the actual shopper. That is extended to Catalina.”

He doesn’t think the trend is toward an opt-in model, “but opt out is definitely an opportunity at all those retailers and so we’re going to have to watch that very closely.”

Because consumers really don’t fully understand how their loyalty data are being used, O’Grady believes it’s incumbent on the industry to educate them. The first step is making things clear. “They’re probably not going to resent it. They resent probably surprises.”

Step No. 2 is give people a chance to opt out of data-sharing, followed by making sure “your partners are being really, really clear and disciplined about how they protect the data. Because any of those three things could derail you very quickly.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“We’re still the best way to get in front of an audience, but it’s changing how advertisers and agencies want to get in front of that audience,” Gianunzio says of broadcast TV in this interview at the recent Beet Retreat 2018 in which he envisions a future where other local media can be sold alongside broadcast.

Videa partners with TV stations to deliver a live, supply-side marketplace for full-schedule, forward-reserve broadcast inventory that can be ordered weeks or quarters in advance. It’s a way of automating the buying process, with pricing approved by the individual stations and second-by-second campaign reporting.

It’s a process designed to “sand off the rough edges of how we buy and sell,” says Gianunzio. The fact that someone in a local broadcast buying department handles thousands of spots is “insane.”

He recalls his time as a sales rep for Cox as the digital age dawned and local broadcast revenues were declining. “And the revenue leaves not because there’s not a good return on investment by running your spots in broadcast. And it’s not because it’s not where advertisers want to be,” says Gianunzio, citing Google and Uber as examples. “The reason advertisers and agencies pull their money out of it is because it is so cumbersome.”

Videa has been building a “foundation” to unify station sales. “Within each station broadcast group we’ve got different ways of doing things. I’m just talking about getting everyone to think the same way” and hopefully agree on activities that should be automated. “Automation scares people because if I’ve been doing it manually, that’s what I’m getting paid for.”

He talks about the complexities brands face when entering a new market and wanting to surround prospective customers with ads across all the various media available and do it quickly. “You’d be discouraged from doing it,” says of the current environment of going seller to seller.

“And that’s when you can see real power in advertising, when you’re surrounding a consumer with each of the forms of media that you’re using so that it’s not a different message coming from each one.”

Videa’s future vision is to be able to unite more media sellers. “The idea is not for a platform like this just to exist for broadcast. It’s to exist for broadcast and everything that comes with it, OTT, VOD, then network and radio,” says Gianunzio.

“If we could build sort of a superhighway where media can exchange with data and exchange with revenue, we’re in a position where you can make a smart decision about opening up in a specific market, you can surround a market and have it all in one place.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

Beachfront tries to get “as close as possible” to audience and content on behalf of its publisher clients, he adds in this interview at last month’s Beet Retreat 2018.

“Everything around our SSP is working on behalf of the publisher and how close we work with the publisher to expose some of that data around the actual video content and what’s happening with the audience to the advertiser to make it relevant,” Sinton adds. “Not just audience relevance but contextual relevance and making sure the content is brand safe and things of that nature.”

On the television side, Beachfront is working with a variety of connected-TV partners at various tiers of the food chain, along with desktop and mobile. “Someone could be watching a piece of content on a connected TV and finish it on a tablet. We want to be able to connect those dots for the advertiser,” Sinton says.

“Right now of course, the big thing everyone is going towards is how do we create more addressable TV. When I say addressable TV I really mean how to create TV that basically has all the things in digital around targeting and be able to deliver on a big screen.”

With regard to video and video content, “We want to make it programmatic and as real time as possible.”

Given that it can take three to five days to traffic a campaign to linear TV, Beachfront would like to reduce that to three to five minutes. Its goal is to “create a much more real-time, biddable environment for this super premium TV audience. That’s the next big phase I think we’re going through over the next three to five years,” says Sinton.

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

In this interview at the recent Beet Retreat 2018, Heftman elaborates on networks’ mediator-like role in trying to please both advertisers and agencies in an increasing complex, consumer-led TV universe.

Brands are talking about business results like topline growth, better metrics and greater ROI. “When we talk to agencies, obviously agencies are intimately involved in that but ultimately agencies are also judged on a cost efficiency basis,” Heftman says. “And so that necessarily creates a bit of friction in the conversations that we have between brands, agencies and ourselves.”

Increasingly, the seller’s role “is being a bit in the middle of that,” he adds. “Making sure that we are delivering the cost efficiency and proper business rationale to our agency business partners, while working with brands to try and convince them that television in particular is a driver of their business. We never really had to do that historically. The agencies took care of that for us.”

He isn’t suggesting that some agencies don’t tell that story to their clients. But given the roadblocks that exist to connecting television to business outcomes, “it’s not necessarily the best decision to let somebody else do that for us. We have to take some of the lead on that ourselves.”

Heftman sees more marketers than agencies leading the way on innovating new forms of ad products, one reason being it’s not a skill set that one necessarily finds within agencies. “Sometimes it is faster to go straight to the client and work with them on something that works for their business.” This is particularly true for newer, direct-to-consumer brands that might not have a legacy relationship with an agency. “They simply have different expectations from television, and part of working with them is that it has gotten us up to speed in talking more about outcomes and different types of creative and different types of executions that are going to get use there.”

He’d like to see two things going forward. First, realistic acknowledgement that big changes aren’t going to happen overnight, and second an understanding that TV serves certain purposes that can be “a little bit dual” in nature. “I would like to see clients and agencies spend more time thinking about their funnels and really being honest about the places where TV fits into the funnel and things that it has to do to succeed there,” Heftman says.

“I think the sooner we get to a place where we are honest about what each different format can do, that’s going to get us closer to where we’re headed.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“Fortunately, thank god the stigma of programmatic TV and those two words being bridged together is not a bugaboo anymore or freaking people out,” the CEO of Prohaska Consulting says during a break at last month’s Beet Retreat.

People recognize now “through education and getting people off the ledge, sellers, buyers and anyone else in between,” what can actually be accomplished. Prohaska cites some of the benefits of programmatic in display, mobile app and premium, pre-roll video “without the real-time nature and it doesn’t have to be a copy paste the exact same way.”

As some of the conference speakers indicated, there’s more cross-pollination of talent in agency teams. “People are starting to bring different skill sets and points of view to the same type of advanced TV buying where context does matter,” Prohaska says. “People are realizing, fortunately, that the benefits of buying in-program and contextual placements do matter if you can bring some audience targeting to the party.”

Still “very much a work in progress” is frequency capping across screens, “obviously not there yet, I think we all experience it as consumers.” He sees “a concerted effort to be able to figure that out both probalistically for now and deterministically where that’s available.”

Transaction friction costs are “not as perfect as they will be,” whether it’s leveraging the OpenAP platform “or any other more direct connections.”

There are still too many touchpoints and a “little bit too much delay between being able to execute a deal and then actually launch it. And then being able to optimize either by message sequencing or just by media mix optimizing and being able to appreciate where the best placements are,” he says.

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

While FreeWheel’s focus has always been on premium video from a variety of publishers, “There’s now a very low threshold for launching a app that will get TV distribution, so there is a big part of the industry now that is long-tail digital video but actually on a TV,” says Smith, who is GM of FreeWheel Markets.

“We spend a lot of time and energy when we’re evaluating who we want to work with to make sure that publishers that we don’t know well go through a pretty exhaustive process,” he adds in this interview at the recent Beet Retreat 2018.  Interviewing Smith for Beet.TV is Matt Prohaska.

Among the items on the inspection checklist are validating traffic and the right to sell inventory to brand safety concerns. “We’ve created a quality threshold that we use for anything that we package up and bring to market. We spend a lot of time educating brands and agencies about that.”

While it’s nothing new to FreeWheel, it’s not a shared regimen.

“We are fortunately enough that we can be fully transparent with what runs in our ecosystem because we have such a focus on quality,” Smith says. “But not everyone in the ecosystem understands that, and if marketers are looking for scale, it’s sometimes easy to trade off scale for lesser quality-inventory.”

Asked whether it’s getting any easier to work with all of the various parties needed to plan and execute campaigns, he feels “we’re getting closer but it’s still very complex.”

Within the major holding companies “it varies pretty significantly. So we invest a lot of time in mapping out those organizations and understanding who the key people are, regardless of title because those titles also kind of vary wildly across holding companies.

“But also ultimately connecting to the teams that work with the brands. Brands are driving the decision-making and the holding companies are trying to serve their needs,” Smith says.

Then there are what he terms “mid-market” agencies that never scaled to the extent of the biggest industry players and whose teams are thus more unified. “The value proposition to those organizations and the level of friction is much lower sometimes than going into the major holding companies. So in some sense we’ve had a lot of success, and I feel like some of those agencies are taking advantage of that unification of linear and digital in a way that the bigger ones are not yet. But I fully believe we’ll get there very quickly.”

Third Quarter Video Stats

Subsequent to this interview, FreeWheel released its Q3 Video Monetization Report. Here are some key findings:

U.S.: Top line ad view growth of 26% year-over-year, fueled in part by 87% ad view growth on IP-enabled MVPD platforms.
U.S.: While the ratio of traditional MVPDs ad views to virtual MVPD ad views on IP-enabled platforms is 80:20, vMVPDs are growing almost twice as fast.
U.S.: OTT devices and smartphones combined now make up 60% of total ad views.
E.U.: There was top-line ad view growth of 16% despite U.S. publisher ad views in Europe declining 21% in light of a cautious approach to the GDPR.

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“The theme for us continues to be to provide audience-driven solutions for marketers, where they can buy on an audience-basis inventory across linear TV, social video, over the top. Everything in one place,” says Chief Revenue Officer Anupam Gupta.

“One of the things that we hear from marketers again and again is the audience problems are getting solved,” he adds in this interview at last month’s Beet Retreat 2018. “That’s pretty much in a good place now. But now they’re looking for more scale.”

4C’s most recent partnerships within linear TV are with a4, a unit of communications giant Altice, whose networks include A&E, CNN, The History Channel and Discovery, and AMC Networks’ MASS Exchange. 4C’s self-service platform facilitates live, real-time access to inventory avails in the scatter market and performance data, as well as audience-driven Upfront allocations.

“So that’s now integrated within 4C where you can start with an audience and you can add that to repertoire of NBCU inventory” that became available to 4C clients via a partnership crafted in 2017, Gupta says.

“Essentially what you’re seeing is more linear networks making their inventory available through the aggregator partnerships and platforms like that are integrating to provide more scale for marketers.”

On the TV-to-consumer side, 4C has new integrations with FreeWheel, SpotX and Telaria. “For us, it’s natural to add over-the-top inventory to our platform. I think the scale issue is being addressed through these collaborations and integrations.”

4C’s three priorities are giving marketers flexibility to use whatever data they wish, help them plan and buy across premium content and drive and measure business outcomes.

“Our goal is to stitch those three pieces together into one platform so that marketers can essentially make progress in the data driven world,” Gupta says.

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

Last month, the two companies launched Direct to Scale, whose core offerings are media strategy and making premium video advertising on linear TV and digital media more accessible to direct-to-consumer businesses, as Campaign reports.

In the Giant Spoon partnership “we are focused mainly on direct-to-consumer brands that they represent, but not only that, the opportunity is really for us to expand our roster of advertisers beyond the current more traditional legacy advertisers,” says Brian Norris, SVP, Audience Sales, NBCUniversal.

In this interview at last month’s Beet Retreat 2018, Norris talks about transitioning from traditional media currencies and embracing more outcomes-focused metrics. He explains how NBCU’s in-house Future of Advertising Group is focused on “changing how we interact with marketers in terms of making sure that ultimately, we’re providing the most ROI for them long term sustainable growth.”

That process starts with providing the best content for consumers that, in turn, produces better results and better environments. “We’re looking at currencies that measure impact and outcomes as opposed to traditional currencies, which ultimately we’re not able tie business results to traditional measurement currencies,” Norris says.

Having started in TV at Lifetime Television in 2003 before moving on to Viacom and then DISH Media Sales and Sling TV, Norris has a broad perspective of marketers’ evolving expectations.

“I suspect that many advertisers will have different KPI’s and it’s upon us to be as nimble as possible to be able to achieve those results for them.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

This is why Prasad reflects on this year’s Retreat in this interview as “a small and purposely put together group like this where there’s a lot of people who are working together or thinking about working together and able to share ideas.” He says he prefers learning from “power panel sessions” than typical one-hour conference sessions “with everyone repeating themselves on a panel. It doesn’t happen here.”

Aside from Puerto Rico’s natural beauty, Prasad notes the Retreat location in the context of the devastation that Hurricane Maria caused in the fall of 2017 and the ongoing rebuilding. One element of the Retreat was dedicated to raising awareness of the short- and long-term impacts on residents and the marketing world.

“It definitely seemed like Andy and the team here really wanted to make sure that we understood that and there was a connection to it,” he says. “When you put that all together, I thought it was a really special few days.”

The time was shared with “so many partners across the ecosystem and clients that we’re working with. Maybe two years ago I would have looked across the room and said it would be great to be doing this with that company, trying to connect this supply source with this data so that this advertiser can execute. And now it’s actually kind of happening.”

Prasad describes “a really busy last two years” in which VideoAmp has been assembling its marketing investment platform, which encompasses data-driven, cross-screen planning and measurement, activation and programmatic. This year, the company acquired TV-data processing provider IronGrid Data Services, which when combined with a partnership with Inscape formed the basis for the new Data & Emerging Products Division, as Variety reports.

“So next is going to be taking all of these pieces of technology and solutions and making sure they’re optimizing to work together so that one piece of our system, like the measurement system, is actually informing how you should be planning,” says Prasad.

He clarifies that it’s not planning to determine a particular dollar spend for a particular medium “but actually event-level forecasting. Which means that we’re also going to be more tightly working with big media companies and suppliers and supply side technology to make that happen.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“The last few years have really seen the transformation in the digital living room, where the TV has gone from being a content generator or content distributor to now really just becoming a device in the living room,” Bindra says.

“For us, we look at the extension of video going onto the TV screen, the largest screen, as something that we have to deliver to our clients to give a more omni-channel output to their campaigns.”

In this interview at last month’s Beet Retreat 2018, Bindra talks about the challenges of working within connected-TV and OTT environments and how the digital side needs to learn from past mistakes when branching out to TV.

“We’ve run into similar challenges as other DSP’s of how do we take our optimization and our reach capabilities that the marketplace really knows us for and recognizes us for and then expand that into this new arena,” Bindra says. “There’s a lot of work for us to do around this.”

Sizmek has been running test TV campaigns, some of which began on the digital side, that have for the most part generated responses he describes as “fantastic.” But outside of test environments, it’s the same story that many others are encountering.

“We’ve run into similar issues of scale, of data on a de-duplicated or a deterministic basis,” Bindra says. “So a lot of our technology, which is very much centered around being precise, now has to start to look at things in a less-than-perfect environment from a data perspective.”

Data access isn’t the problem, nor is figuring out who a particular viewer is since everything is at the household level, “but also how attribution works and the time delay in getting that information back while the campaign is running,” he adds. “Our focus has really been in how we can collapse that time gap and who we should partner with to get data back to us in a near real-time fashion.”

Asked to reflect on Beet Retreat 2018, he says it was “extremely helpful” being around knowledgeable practitioners who are open about discussing issues common to many participants. “Being relatively nascent in this space, this access was very valuable to us. There are things that you just don’t know where to start and who to talk to and how to evaluate things,” says Bindra.

Another common sentiment at the Retreat was a willingness to solve, according to Bindra. “In the digital side, we have made mistakes and we are now acknowledging those mistakes, we are trying to learn from those mistakes.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

In this interview with Beet.TV, the VP of Client Partnerships explains how NCC leverages its MVPD unique subscriber identifiers matched to set-top box data and then marketers first- and third-party data “not just for addressable that NCC Media is selling but across their national TV investment.

“So when you start to see how your specific first-party audiences are being exposed across those huge TV video channels, it opens up a lot of possibilities to start to understand reach, frequency the best ways to make the plans to reach your audiences more effectively and efficiently,” Seth says during a break at last month’s Beet Retreat 2018.

One topic that’s starting to come up is how to consider audiences to help with advocacy and loyalty. “You of course aren’t going to build an audience segment against something for advocacy, because that’s going to be a really small audience,” Seth explains. “The returns on that aren’t going to be efficient.”

The way to make addressable work for big national brands is to start building audiences with sub segments with each of the funnel steps in consideration. It involves using data combined with brand health information “that can indicate that it will eventually drive to a different behavior, or to the behavior you want, and how do you start to layer in building those audiences to serve up more robust audiences across screen.”

The “cool thing” about that longitudinal approach is measuring it across time with the help of partners in the multi-touch attribution space. NCC uses Data Plus Math to trend data “so that you really see how you’re driving that path to purchase or that path to advocacy.”

Seth cites the example of building an audience using an audience that has visited a website and then layering on subset of lookalike audience to website visitation. When measured over time, “you can see pretty immediately that the website audience was driven to conversion more than the audience that was the lookalike model. But the lookalike model was driven to the website and so you can start to see how you can robustly build that out to track how your efficiently moving that consumer through the funnel and hopefully it’s more than just than just your advertising plan.

“You’re starting to leverage that data in a way that helps impact your creative strategy, your platform strategy, the ad load strategy and such.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

Along with Vizio and its data unit Inscape, the consortium’s members are AMC Networks, CBS, Discovery, FreeWheel, Hearst Television, NBCUniversal, The Walt Disney Co. and Xandr, as Reuters reports.

In announcing the consortium, Vizio said that TV manufacturers use different technology and standards to enable addressable advertising. “It creates a level of complication for (TV networks), and scale is critical,” Reuters quotes Inscape SVP Jodie McAfee as saying.

While Vizio will create the new technology, it will be an open “industry standard” that competing TV makers can integrate into their products.

Last December, Beet.TV interviewed McAfee at Beet Retreat 2018 about Inscape’s experience in dealing with the Federal Trade Commission on issues involving consumer privacy and viewing data collection. In light of the news regarding Project OAR, Beet.TV is republishing that interview.

Being a pioneer has always carried risks and rewards. Smart-TV manufacturer VIZIO’s Inscape data unit found this out when the Federal Trade Commission a couple of years ago first started looking into what happens to viewer data collected with automatic content recognition, says Inscape SVP of Sales & Marketing Jodie McAfee.

Interacting with the FTC “was a little bit of a bad news, good news situation for us,” McAfee says in this interview at the recent Beet Retreat 2018, where one panel discussion was devoted to data and privacy.

“The bad news was we were the first smart TV manufacturer to actually have such a discussion with the FTC. The FTC had never really taken a look at smart TV data collection and privacy regimes around it.”

The good news was “the fact that it put us in a position to actually sit down with the FTC and say, ‘okay, what do you want to see? What do you think a privacy regime should look like on a smart TV? So it became more of a collaboration as opposed to the FTC simply throwing down some ground rules and walking away.”

Nonetheless, the FTC fined Inscape for inappropriately collecting viewing data from 11 million TV sets.

“That back and forth generated really almost what our industry views as the VIZIO standard for smart TV data collection,” McAfee says. “And it also prepared us in advance for a little bit of what’s going on with GDPR because there were certain things around what the notification needs to look like, how it needs to read, that are now part of GDPR that we had already done coming out of that FTC negotiation.”

The FTC believed that many people are so intent on getting through the initial setup of a smart TV, they just scrolled through the terms of service so they could start watching something quickly, according to McAfee. “So they said what we’d like to see is a data collection notification that is separate and prominent from the rest of the terms of service.”

Thus was born VIZIO’s “completely separate screen” during the setup process titled “TV viewing data. And it walks through here’s how we do it, here’s what we’re doing, here’s who we give it to and here’s what they do with it.” People can either agree or decline, meaning data collection is off by default.

“So it is a true opt-in regime. We pushed that new regime to our entire footprint of TV’s in February of 2017 so we’ve been fully FTC compliant ever since,” says McAfee.

Changing its policy and communicating the new one to existing owners via a firmware update cost the company 2 million TV’s from its overall footprint because older ones “didn’t have enough processing power to actually support the user journey when we decided to make the opt out process easier.”

Have all other TV manufacturers taken note and now follow the FTC mandate? Not from what Inscape sees when the company checks them out, according to McAfee, most likely because they’re not in the data business like Inscape.

“The process for pushing a firmware update to a TV, it’s a pretty meaningful exercise. So I’m sure other manufacturers just did the math. Is it worth it or not. And I think some of them went, ‘not really.’”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“It’s like de facto. And that’s pretty cool. Maybe iSpot should get our crown pretty soon, because we’re rapidly becoming a really trusted third-party analytics provider for TV advertising,” the SVP, Business Development, says in this interview.

iSpot.tv started as a measurement company that could track TV ads in real time, according to Bareuther. “We built a syndicated product that measured TV ads and it became really popular for brands.”

About four years ago, the company started a partnership with smart TV manufacturer VIZIO to get access to ad impressions as they appear, based on automatic content recognition.

“So we’ve taken all this raw data from what’s appearing on screens second by second and do a lot of data work and data science around that,” Bareuther says.

There’s a range of information gleaned, including whether the ads are being seen live, via DVR or OTT, plus the pods and the shows in which the ads appeared. A typical analysis will include the number of airings of a spot, the estimated advertiser spend, the number of impressions, attention and engagement scores and percentage of share of voice.

The next step was being able to show what happened after viewers had seen the ads, as happens with much of digital advertising.

“Brands are spending so much money on TV and they’re spending so much money on digital and they know what’s happening in digital immediately. They get long feedback loops from TV,” Bareuther says.

iSpot.tv set out to shrink that feedback loop. “So we figured out a way to understand household consumption of television ads” so that advertisers could blend it into their marketing stacks, through LiveRamp, Adobe, Oracle et cetera. And help them understand themselves how it’s being effective.”

Asked whether any of its clients have enough outcomes data to be able to use it not just for back-end measurement but for planning purposes, Bareuther says yes.

“In order to get to the bottom of the funnel the first step is always to have the top of the funnel. Once we can say that this NBC show is more effective for you X retailer than this ABC show or this Bravo show or vice versa, maybe could use it to optimize your planning.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

When brands bring data to Comscore, it matches that information through privacy complaint manners to the viewing behaviors generated by its cross-platform data sets. “I think that is the misnomer of when people say ‘I want to bring this first-party data. You have to then have all of the rights in order to match that,” Hinnant explains in this interview at the recent Beet Retreat 2018.

“I do think that is something that needs to be verified and Comscore plays a big role in that we can make sure that it’s matched in the same way,” Hinnant adds. “We can verify to any agency or brand that wants to come in that this is how we match the data and this is the sizing of that particular segment and this is what was delivered.”

Asked for her thoughts about OpenAP, the audience targeting consortium created by Fox, Turner and Viacom, she sees it as one of the answers to the industry challenge of removing friction in the marketplace. Comscore’s advanced audiences was one of the first data sources integrated into the new system.

“I think that the partners have done a good job to pool their inventory and allow any marketer to come in and say, ‘Yes this the segment I want to trade on and this is the pool of inventory.’ So it does make that process easier.”

According to Hinnant, advertisers ranging from American Express to Universal Pictures are trading in the OpenAP system, “so I think those are great at-bats and great starts. I think there needs to be more inventory available. Comscore’s just happy to be playing in that universe.”

To thread the needle of varying players all seeking to harness more data for consumer targeting, Comscore is “beholden to all of our verticals. The MVPD’s have put us in business through their delivery of data that we manage for them, set-top box and return path data,” Hinnant notes. “We look at all of our partners across the digital ecosystems and then our own panel information” to deliver a cohesive measurement system that’s useful. We’re trying to help provide the solutions to our customer base.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

In this interview, Swartz, a longtime Beet Retreat attendee and unofficial dean of the proceedings, sums things up thusly: “We’re trying to race F1 with a Datsun.”

Furious Corp. was founded in 2013 to provide linear TV and video yield optimization. It’s seen recent transactional data from the trafficking systems of operators and local station groups showing anywhere from five to 15X premiums on ad pricing, according to Swartz.

“Right now is an interesting time because although we’re talking about the challenges and the fear, uncertainty and doubt in TV, we’re coming off one of the best political seasons we’ve had,” she says.

That’s the good news. The not-so-good news is that it’s typical for clients of Furious to be using “software that was released in 1990. They’re running their ad businesses with Excel.”

As a result, the desire and need to move things forward is hindered by friction in the workflows of revenue and inventory management, causing revenue leakage. “The driver of revenue leakage is different depending on the TV product you’re selling, and whether you’re using a Nielsen guarantee for spot or impression-based sale,” Swartz explains.

With traditional, spot-based linear, the greatest revenue leakage is driven by pricing variability, implies inefficiency. Bringing uniformity and consistency to the rate card process “is in incredible opportunity to improve revenue” by three to 10% in a 12- to 24-month period, Swartz says.

For impression-based, addressable ads, “It’s actually about inventory utilization and allocation, and so the business challenge changes. The need to look at the other level of yield, which is inventory management, as a different part of that workflow to focus on in order to ensure that, as you’re introducing new products, you still continue to utilize your assets and inventory as efficiently as possible.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.

“I think it’s a wildfire right now,” Lutz adds in this interview at last month’s Beet Retreat 2018.

She sees more agencies wanting to use TiVo’s MVPD and box viewing data to create targets for viewers of connected-TV and non-linear devices “and export that out as part of a segment.”

It could be in combination with data about, say, politics or location. Based on viewing behavior, ads for a new horror movie could be targeted to people who are known to be theatre-goers and horror aficionados.

Then there is the concept of exposed and unexposed ads and how to achieve desired frequency and measure it. TiVo can help to determine how to retarget people who have been exposed to specific ads or help to find “those viewers who I wanted to target but are light TV viewers and I didn’t actually find them on TV,” Lutz explains.

TiVo’s second-by-second viewing data also provide insights into how to retain audiences and how effective efforts are to improve the commercial experience by adjusting pod lengths. While there’s planning, measurement and attribution, “they’re all very much in parallel. If I plan, I create my target, I will most likely model that target today because I want to get the scale that I need to find those viewers on platforms.

How does one determine whether a particular target is effective?

“The only way to know that is to start looking at attribution. During the course of that campaign, of that model segment who converted and look at how those converted viewers score against that model target. Was that model effective?

“There are so many different questions and measurements and evaluations that need to happen that are just iterative across a process,” Lutz says. “I don’t think you can look at things in a silo.”

This video was produced in San Juan, Puerto Rico at the Beet.TV executive retreat. Please find more videos from the series on this page. The Beet Retreat was presented by NCC along with Amobee, Dish Media, Oath and Google.