Brain Typing & Skin Hearing: Everything You Need to Know About Facebook‘s 2017 F8 Conference

Every year, developers and tech industry insiders descend upon Silicon Valley for Facebook‘s annual F8 conference. While F8 tends to focus heavily on technical topics geared toward the developer community, it has also become the stage where Facebook often unveils its most ambitious and forward-looking projects that have implications far beyond the world of coding.

At the 2017 edition of F8, held in San Jose on April 18-19, Facebook did not disappoint. Along with a bevy of updates to core products like Messenger and the Facebook camera, the company pulled back the curtain on some truly mind-blowing (literally) technologies coming out of its Building 8 research lab.

But before we dive into brain-computer interfaces and skin hearing, let‘s recap some of the key themes and announcements from the event. Consider this your comprehensive guide to everything that went down at F8 2017.

The 10-Year Vision: VR/AR, AI & Connectivity

In his opening keynote, Facebook CEO Mark Zuckerberg laid out the company‘s product roadmap for the next 10 years. Three major pillars underpin this vision:

  1. Virtual & Augmented Reality – Facebook sees VR/AR as the next major computing platform that will redefine how we interact with digital content and each other. The company is investing heavily in hardware and software to bring this vision to life.

  2. Artificial Intelligence – AI already powers many of Facebook‘s products, from News Feed ranking to photo tagging. Facebook is doubling down on AI research and applying machine learning across its family of apps to make them smarter and more personalized.

  3. Connectivity – Facebook‘s mission is to bring the world closer together, and that starts with its connectivity efforts to provide internet access to the 4 billion people who currently lack it. From solar-powered drones to new wireless technologies, Facebook is exploring multiple avenues to connect the unconnected.

Throughout the keynote and subsequent sessions, Facebook executives dove deeper into the progress being made on each of these fronts. Let‘s unpack the key updates.

The Future is Virtual (and Augmented)

It‘s no secret that Zuckerberg believes virtual and augmented reality will play a huge role in shaping the future of computing. Facebook first signaled its big bet on VR with the $2 billion acquisition of Oculus in 2014. Now, with VR headsets becoming more affordable and accessible, Facebook is making a concerted push to accelerate adoption.

At F8, Facebook announced Facebook Spaces, a new VR app for Oculus Rift that allows you to hang out with friends in a virtual environment. You can create a personalized avatar, watch 360 videos, draw 3D objects, and even take virtual selfies. While still a bit gimmicky, Spaces offers an early glimpse at how social interaction may play out in VR.

But where things really start to get interesting is when you combine the immersion of VR with the real-world utility of AR. To that end, Facebook unveiled its Camera Effects Platform – a set of tools for developers to build augmented reality features into Facebook‘s in-app cameras.

Similar to Snapchat‘s popular Lenses, the platform allows for the creation of masks, filters and 3D effects that can recognize and interact with real-world objects. So imagine being able to "try on" virtual sunglasses, conjure up a 3D hologram in your living room, or leave a digital Post-It note for your friend at a specific location. With Facebook‘s enormous reach (the company now has over 5 million active advertisers), expect to see a lot more branded AR content popping up in your News Feed.

Facebook also showcased some far-out AR/VR projects in the works, like the ability take a virtual "selfie" with a friend‘s avatar and post it to your (real) Wall, or use 3D mapping technology to let a friend join you on a virtual tour of your home from thousands of miles away. While these demos may seem like novelties, they hint at a future where the boundaries between the digital and physical start to blur.

Teaching Machines to Think

The other major theme that emerged from F8 was just how much artificial intelligence is being baked into everything Facebook does. From visual search in the Facebook app to smart chatbots in Messenger, AI touches almost every corner of the company‘s ecosystem.

One of the big AI announcements was the launch of Caffe2, a new open-source deep learning framework developed by Facebook. Caffe2 is designed to make it easier for developers to deploy AI-powered features and services that can run on mobile devices.

So why is this important? As impressive as Facebook‘s AI capabilities already are, most of that intelligence still lives in the cloud – reliant on powerful servers to crunch all that data. With tools like Caffe2, Facebook is essentially trying to "democratize" AI by bringing more of that processing power directly to smartphones and other consumer devices.

Imagine a Facebook camera app that can instantly recognize any object it sees, or a Messenger bot that can engage in a freeform conversation without constantly pinging a server. That‘s the kind of on-device AI Facebook is aiming for, and it could dramatically reshape how we interact with technology.

Of course, with all this AI power comes great responsibility. Facebook has already come under fire for its algorithmic approach to the News Feed and ability to influence what information people see (or don‘t see). As the company embeds AI even more deeply into its products, it will need to be transparent about how these systems work and put safeguards in place to prevent unintended consequences.

But overall, Facebook‘s AI push is a clear sign that the company believes intelligent software will be the foundation upon which all future services are built. From self-driving cars to digital assistants, AI is poised to reshape entire industries – and Facebook wants to provide the tools to power that revolution.

Typing With Your Brain, Hearing With Your Skin

If Facebook‘s 10-year roadmap reflects the company‘s long-term bets, the moonshot projects coming out of its Building 8 lab offer a more fantastical vision of the future.

Helmed by Regina Dugan, the former head of Google‘s Advanced Technology and Projects (ATAP) group, Building 8 was created to develop breakthrough hardware innovations to advance Facebook‘s mission of connecting the world. And based on the projects unveiled at F8, it‘s clear the team is swinging for the fences.

The headliner was a "silent speech" interface that would allow people to type using only their thoughts. As Dugan explained on stage:

"What if you could type directly from your brain? It sounds impossible, but it‘s closer than you may realize."

The goal is to develop a non-invasive, wearable sensor that can measure brain activity and translate it into digital commands, allowing you to "type" at a rate of 100 words per minute – 5X faster than average typing on a smartphone.

Facebook is quick to point out that this isn‘t about reading minds or decoding inner thoughts. Rather, it‘s about capturing the words you‘ve already decided to share and sending them to the computer or phone, without the need for fingers or voice. Still, the fact that Facebook is even talking about a "brain mouse" shows just how far the company is willing to push the boundaries of human-computer interaction.

But that‘s not the only sci-fi project in the works. Building 8 is also developing a way for people to "hear" language through their skin. The system works by delivering vibrations at different frequencies to the arm, allowing the brain to interpret those signals as specific phonemes or words.

In a live demo at F8, an engineer was able to distinguish between three different tactile inputs she received simultaneously: "Nice to meet you", "How‘s it going?", and "Congratulations". The technology is still in the early stages, but Facebook envisions it one day being used as a new form of highly private communication, or as an accessibility tool for the millions of people with hearing impairments.

When you step back and consider the implications of a world where we can type and communicate using only our minds, or "feel" words transmitted through our skin, it starts to sound like the stuff of a Black Mirror episode. And indeed, these projects raise a host of ethical questions around privacy, security, and what it means to be human in an age of increasing symbiosis with technology.

But those are precisely the kinds of big, hairy, audacious challenges Facebook seems eager to take on as it looks to shape the next era of computing. With initiatives like Building 8, the company is sending a clear message that it intends to be a leader not just in software, but in hardware and interfaces that quite literally get under our skin.

The Bottom Line

So what should marketers and businesses take away from all the shiny new toys and head-scratching experiments trotted out at F8? Here are a few key points:

  • AR is the new QR – Just as marketers once rushed to put QR codes on everything, expect a similar land grab around AR experiences. With tools like Facebook‘s Camera Effects Platform making it easier to create branded filters/lenses, savvy marketers will be looking for ways to integrate AR into their campaigns. The key will be to focus on utility and engagement vs. just novelty.

  • Chatbots are growing up – Facebook Messenger now boasts over 100,000 monthly active bots, and with new features like chat extensions and AI-powered suggestions, the platform is becoming a more robust channel for customer service, content delivery, and transactions. Brands should be exploring how conversational interfaces can enhance their marketing mix.

  • AI is the new mobile – Just as mobile was the mega-trend of the last decade, AI and machine learning look primed to reshape entire industries over the next 10 years. From smarter CRM to predictive analytics to intelligent assistants, every business will need an AI strategy to stay competitive. Facebook clearly intends to be a key player in providing the tools and platforms to power that AI-driven future.

  • The battle for our bodies is just beginning – While Microsoft has HoloLens and Apple is rumored to be working on smart glasses, Facebook staked its claim as a leader in the "full stack" of VR/AR capabilities. From the Oculus headset to AR development tools to far-out projects like skin hearing, the company is positioning itself to own the technology that will mediate our experience of the physical and digital worlds. Marketers will need to watch this space closely.

At the end of the day, the biggest takeaway from F8 is simply that change is the only constant. Facebook‘s relentless pace of innovation is a reminder that the platforms and paradigms we‘ve grown accustomed to can and will evolve more quickly than we often expect.

As Zuckerberg stated in his manifesto earlier this year, Facebook‘s job is to "[keep] our society moving forward". Based on the vision laid out at F8, it‘s clear that the company has no intention of slowing down in its mission to connect the world – and the human brain – like never before.

It‘s up to the rest of us to make sure we‘re ready for the ride.

What were your main takeaways from Facebook‘s F8 2017 conference? How do you think these cutting-edge technologies will shape the future of marketing and communication? Leave your thoughts in the comments below.