• The Neuron
  • Posts
  • ๐Ÿˆย Open source beats Microsoft to the punch

๐Ÿˆย Open source beats Microsoft to the punch

PLUS: Another lawsuit, transformers everywhere

Good morning! You're reading The Neuron. An average NFL game lasts 3 hours and 12 minutes, 64x longer than it takes for you to get caught up on AI (3 minutes).

Today in AI:

  • Open Source Clones Microsoft Model In 11 Days

  • Getty Images Also Wants to Sue

  • Chart: The Transformer Explosion

  • Around the Horn

  • Leo Sends His Regards

Open Source Clones Microsoft Model In 11 Days

That was fast.

Earlier this month, Microsoft released a paper talking about VALL-E, a model that can clone your voice with just a 3-second clip of your voice.

It's been just 11 days, and it appears a Hong Kong-based software engineer has already used that paper to develop an open-source version of VALL-E. We could get the open source clone before the Microsoft one.

It's a trend. OpenAI's GPT-2 took 6 months from first release to be cloned open source. GPT-3 was 10 months. Led by orgs like Stability AI, the community is moving fast to develop open source versions of everything that a company doesn't release.

But, not everything can be cloned. GPT-4, for example, is likely costing low hundreds of millions to produce. Open source doesn't make that kind of cash (or any at all).

Wondering who will get rich from AI? The battle between open and closed source is an important one to watch. Whoever keeps a bleeding-edge model to themselves can charge money for it (see: OpenAI).

Getty Images Also Wants to Sue

The Year of the Copyright Lawsuit.

Artists sued Stability AI, Midjourney and DeviantArt. Getty Images wants in on the action.

Getty Images says Stability AI scraped millions of images from them. Which is likely true: one estimate puts it at ~3 million of the 2.3 billion images that Stable Diffusion trained on.

We won't go through the arguments again, but there are two points to note:

  1. Stable Diffusion is open-source, and its parent, Stability AI, open-sources everything. OpenAI does not and refuses to share where they got their training data from. Stability AI is getting sued, OpenAI is not.

  2. It doesn't look good when Stable Diffusion spits out the Getty Images watermark all the time:

More on the way? Stock photo sites like 123RF, PhotoShelter, Adobe Stock and Shutterstock are just a few of the other major training data sources. Who's helping Stability AI with those legal fees?

Chart: The Transformer Explosion

How a key finding from Google unlocked everything.

If there's one AI paper title that everyone should know, it's "Attention Is All You Need". It's a 2018 paper from Google that introduced the transformer, a key piece of AI model architecture.

TL;DR: theย transformer is enabling a LOT of cool stuff: ChatGPT, Stable Diffusion, DALL-E, you name it.

Here's a visual of the AI models that use transformers and when they were released:

Around the Horn

DM me links on Twitter: @nonmayorpete

Are you new to all this AI stuff? Here's The 3-Minute Guide to Slaying Your Dinner Convo About AI to get you up to speed. Or at least smart enough to impress your family.

Leo Sends His Regards

That's all we have for today. See you cool cats on Twitter if you're there: @nonmayorpete

What'd you think of today's email?

Login or Subscribe to participate in polls.