Sunday , 24 January 2021
Breaking News

Inside Netflix we reveal how brain scans and bots help shows go from lens to your living room

A mural in the newest building at Netflix's Silicon Valley HQ features characters from its original series

Netflix first launched in the UK in 2012 and, along with catch-up services like BBC iPlayer and streaming rivals like Amazon Prime, has completely transformed the way we watch television.

WIRED was invited along to the firm’s recent Netflix Labs Day at its Los Gatos headquarters in the heart of Silicon Valley for the global release of Marvel’s Iron Fist and to hear more about the innovations that brought it to the screen.

The firm is renowned for its ever-expanding range of original series from political drama House of Cards and 80s sci-fi throwback Stranger Things to 13 Reasons Why – one its latest offerings telling the disturbing story of why a teenaged girl took her own life. Unlike Amazon, Netflix has ditched the expensive process of producing pilot episodes, opting for a more direct approach.

“It really starts with a great idea, and a team wanting to bring it to life,” explained Cindy Holland, VP of Originals Series at Netflix. “We use data to work out what’s the minimum threshold audience size that we need, in order to justify the economics of a project that we’re thinking about”.

Marvel’s Iron Fist is one of the latest arrivals, with the comic book brand’s global clout helping Netflix conquer countries where it’s not so well known. It’s also the first of Netflix’s Marvel series to be shot using Dolby Vision – the audio and video firm Dolby’s enhanced version of HDR.

A simulation of SDR vs HDR output for Marvel's Iron Fist

Netflix has been pushing the use of HDR (High Dynamic Range) for a while now. The technology improves the contrast, colour range and depth of onscreen images dramatically and aims to reproduce images in the most realistic way possible, similar to what the human eye actually sees.

Netflix supports the industry standard version of HDR, along with Dolby Vision. The latter enables frame-by-frame information captured during filming to be preserved as metadata. This is then decoded by Dolby Vision-equipped TVs and translated onto the screen, just how the director intended.

Dolby Vision already appears on several LG TVs, and will soon be added to panels from other major tech firms including Sony and Loewe. What’s more, the tech standard is gradually rolling out to smaller screens, with the LG G6 unveiled as the very first smartphone with Dolby Vision earlier this year.

Just as the director intended

“The goal here is to maintain the integrity of the original content,” said Tony D’Amore, senior colourist at Encore, the man responsible for tweaking the look of shows like Iron Fist. Encore is part of Deluxe, a film company that provides post-production services for many of the big US networks, including HBO, Showtime and FX. D’Amore gave us a fascinating demo of how he used Dolby Vision-powered software on a BlackMagic colour-correcting console to perfect the hues on Iron Fist.

Colourist Tony D'Amore working on Iron Fist

Each of the Marvel shows has its own distinct colour palette from the purple hues of Jessica Jones’ Hell’s Kitchen and the vintage yellow highlights of Luke Cage’s Harlem to the opulent whites and golds of Iron Fist. Though the latter was produced in Dolby Vision from the start, the earlier series have been retroactively remastered.

To answer the question that many Marvel fans have posed, Karim Zreik, senior VP of original programming at Marvel confirmed: “The Marvel cinematic universe is different than the television universe. We will never cross-pollinate. They have their characters, and we have ours.” So don’t expect to see Daredevil teaming up with Spider-Man anytime soon.

While the HDR-based tech opens up new doors for the creative brains behind these series, it also represents a new challenge for directors, similar to the transition from analogue filming to digital techniques.

Iron Fist director of photography Manuel Billeter, the man responsible for devising those distinctive colour schemes, said: “Having Dolby Vision HDR at our disposal is a very enticing prospect, and it’s exciting to explore. It was a bit of a learning curve as well”.

Dolby’s aim is a consistent picture – being able to deliver whatever the director had in mind, on any screen size. As well as helping to convey the look the director intended, Dolby Vision also helps companies like Netflix future-proof their content. The enhanced HDR standard goes above and beyond by catering for TVs with a massive 10,000 nits of brightness. Currently, top-of-the-range TVs only have around 4,000 nits, with most sets limited to 1,000. Furtehrmore, Dolby Vision has been developed to stream more smoothly than conventional HDR by using about 10 per cent less data.

Just like its key streaming rival, Amazon, which also uses HDR and Dolby Vision, Netflix is also producing 4K content to further future proof its shows. However, unlike Amazon Prime viewers, Netflix subscribers have to pay extra to watch in 4K.

Measuring viewer reactions

Dolby’s TV innovation doesn’t stop there. We were given a rare glimpse inside the firm’s biophysical lab, located at its headquarters in San Francisco. Here, researchers scientifically monitor viewer reactions to what they’re watching to give them an insight into how we experience TV.

To do this, volunteers are fitted with EEG sensors to measure electrical activity in the brain, along with heart rate, pulse and sweat-based stress trackers, while a thermal imaging camera is used to detect facial temperature changes which can indicate stress, engagement or arousal.

A viewer wearing EEG sensors watches TV in Dolby's biophysical lab

Going global

The global nature of Netflix – which now streams in 190 countries, with the notable exceptions of China, Crimea, North Korea and Syria – means language presents a huge challenge and the company goes to extraordinary lengths to get it right.

Last month, Iron Fist launched in 20 languages, with 572 “language assets” which include subtitles, dubs, and audio description. To put that in perspective, when Lilyhammer launched in 2012, it had just seven languages and 96 assets. The firm now aims to add two or three new languages every year.

Netflix works directly with the best translators possible and has even developed a unique online test called Hermes to identify them. The test quizzes participants on tricky translations of idiomatic phrases into their target language and will eventually use the metrics gathered to recommend the best subtitler for each specific show.\

Netflix’s director of content localisation and quality control, Denny Sheehan, explains the importance of getting the subtleties of translations spot on, using the Baz Luhrmann-produced hip-hop drama The Get Down as an example.

“The episode titles of The Get Down were actually quotes from Rumi poems so we had to make sure that we actually had the translations of these poems as they originally existed to make sure that we’re imparting them correctly,” he said.

What’s more, Netflix scours Twitter for complaints about its translations, using algorithms to identify and fix them as quickly as possible, sometimes even on the same day. This empathy with global audiences is something that sets Netflix apart from some US-centric tech firms. The Silicon Valley-based company recently announced its first original series to come from Germany, Italy and Spain while Amazon Prime launched its very first German-language series globally last month, dubbed into numerous languages.

Launch night

Once the content is ready, it’s pushed out to local servers around the globe during off-peak times, using Netflix’s custom-built Open Connect system. The tech can optimise the encoding of the content per shot, rather than per title, so that bandwidth is used as efficiently as possible.

“We want to make buffering a historic relic like where your kids say to you, ‘what’s buffer?’”, said Netflix CEO Reed Hastings.

For the midnight (Californian time) launch of Iron Fist, Netflix assembled a mission control-style ‘War Room’ at its Los Gatos HQ. As the clock counted down the seconds to launch, teams responsible for each area, including mobile and TV, confirmed in turn that they were good to go. As soon as the new series dropped, and Champagne corks were popped, live stats were displayed on screen showing a steadily growing stream of global viewers firing up the first episode. Netflix declined to divulge the exact numbers.

Next-gen television

Having pledged to spend $1billion (£8m) on technology this year – plus a cool $6bn (£4.8bn) on original content – Netflix continues to innovate. A fair slice of this cash goes towards developing the personalisation of its user interface. Earlier this month, it ditched its ambiguous star rating system in favour of a thumbs up or down rating. While many users were under the impression the stars, which had been used since day one, were an average user rating, they were actually tailored to each viewer based on their viewing habits and any ratings they’d given.

Inspired by the matches given on dating sites, the new system gives users a personalised percentage match with each film and TV programme and is designed to be less confusing and more intuitive.

[Source:- Wired]