As the weights have leaked anyway;

one great move that FB could have done would be to ride the wave of positive PR + get all investors hyped:

---> "Meta is a credible alternative to OpenAI, the company is switching from "Meta"-bullshit to an "IA"-first company",

and get the investors to pump the Meta stock,

and then dilute some of the shares to raise some cash (or issue new shares to newly specialized IA hires).

But no, FB is still going after the VR gimmicks and NFTs.

4 billion USD per quarter wasted on Oculus (!), while they could use this money to fund and support a whole ecosystem around LLaMA.

Meanwhile, the pull request on the official repo linking to a torrent of the weights is still up. https://github.com/facebookresearch/llama/pull/73/files

This seems like a big misstep by Meta. I had assumed they were intentionally allowing that torrent to float around, and tacitly encouraging open source to build on their models. It seemed like a way to differentiate themselves from "Open"AI, and I was actually feeling some good will toward them for a second!

Is there precedent on model weights being copyrightable in the first place? I suppose the recipients of the DMCA notices are unlikely to be willing to contest it in court, though.

Isn't llama-dl just like a human reading information on the Internet and learning from it, only faster?

That is what we have been told when they stole our open source software.

Funny how these bigcorps want to protect the copyright of their copyright infringement machines. Can't wait for someone to train a new model off of theirs and challenge them in court over it. Everything you can slurp off of the internet is fair game, right?

Wouldn't opensourcing the weights and trying to commoditize LLMs be a smarter move? That seems to be the only chance to catch up with OpenAI.

Trying to stop the spread of LLaMA with DMCAs is an interesting technique. Like the internet version of trying to put out a grease fire with water.

The claim is slightly misleading since it seems that they are going after repos that distribute Llama weights without going through the official channels.

However, that could be an unfortunate corrolary of the fact that in the US if you do not enforce your IP you give up your rights over it. Overall, giant lose-lose, and I wish there was a truly open source model to build on top of.

Hi all. I'm the author of the tweet.

My goal was merely to warn everyone in the LLaMA community that Facebook appears to be trying to shut down the ecosystem that sprang up around LLaMA since the beginning of March.

For a bit of background, I created llama-dl on March 5.

Show HN: Llama-dl - https://news.ycombinator.com/item?id=35026902

Announcement tweet - https://twitter.com/theshawwn/status/1632238214529400832

Since the repo is now offline, you can find an archived version of the README here: https://archive.is/7t3it

The intent with llama-dl was to kickstart an open source movement related to LLaMA. If you're curious about my personal motivations for this, I did an interview with The Verge about that: https://twitter.com/theshawwn/status/1633456289639542789

Over the next two weeks, llama-dl grew to 3k stars, and (according to my bucket metrics) distributed 4M files. Thanks to the availability of a reliable, high-speed download link to LLaMA, other hackers were able to launch projects such as Dalai:

Dalai: Automatically install, run, and play with LLaMA on your computer - https://news.ycombinator.com/item?id=35127020

Dalai has been making headlines all over the place, and especially on ML tiktok. (ML tiktok is surprisingly interesting.)

When Facebook knocked llama-dl offline via DMCA on the 20th, my primary concern was to ensure that Dalai stayed up. After all, the whole point of llama-dl was to encourage the creation of a "killer app" such as Dalai.

After a quick huddle with Dalai's author @cocktailpeanut via Twitter DM, they launched a decentralized distribution mechanism for LLaMA, powered by bittorrent: https://twitter.com/cocktailpeanut/status/163903613304778342...

This ensures the availability of LLaMA in the short term. However, there's a broader issue at stake.

The question is whether model weights themselves can be copyrighted. It might seem obvious that since compiled binaries can be copyrighted, ML models should also be able to be. But the U.S. Copyright Office recently denied copyright to AI generated outputs: https://www.smithsonianmag.com/smart-news/us-copyright-offic...

> Both in its 2019 decision and its decision this February, the USCO found the “human authorship” element was lacking and was wholly necessary to obtain a copyright, Engadget’s K. Holt wrote. Current copyright law only provides protections to “the fruits of intellectual labor” that “are founded in the creative powers of the [human] mind,” the USCO states.

If the model output isn't copyrightable, is the model itself copyrightable?

It's an interesting and important question, and answering it in court is a necessary step. The outcome will determine how models are treated over the next decade.

Now, all that said, Facebook is proceeding under the (untested) assumption that LLaMA is copyright Meta. If that assumption is correct, then they're well within their legal rights to issue these DMCAs. Llama-dl was little more than a bash script pointing to a download link, yet that's sufficient grounds for DMCA, since the whole point of llama-dl was to circumvent a copyright protection mechanism.

My overall goal here is to simply bring awareness to all of these issues. We're entering an era of closed-source ML. I think the history of computing shows that open source is generally a better bet.

Facebook, if you're reading this, I urge you to reconsider your approach. You had the opportunity to gain an incredible amount of momentum. By killing it off, you're sacrificing your foothold into the hearts and minds of ML hackers. Wouldn't it be a better idea to harness the ecosystem rather than stomp it out of existence? There are so many ways this can facilitate your business in a positive way. Are you sure that being an adversary to your own community is the best way forward?

I was thinking about this the other day and assumed they would just relicense it. Guess I was wrong. What's the point? Streisand effect ensures it will be the most sought-after model while they're making a big deal out of it, and its value is going to depreciate quickly due to the overall pace of AI

It kind of doesn't matter. Llama is great because it lets everyone experiment, optimize and learn with LLMs locally. Once Stability releases their model everyone will just drop llama and quickly apply all their new tools and learnings to that model.

For context, Facebook's LLaMa weights were leaked a few weeks ago: https://news.ycombinator.com/item?id=35007978

Pretty interesting that everyone here is overwhelmingly against Meta for trying to keep their models private, but the other company (the one with "open" in their name) is applauded for doing exactly this.

My first thought was “I wonder if Alpaca 7B could generate a plausible counter-notice”. Probably not, but I bet if GPT-4’s weights were leaked it could defend itself against DMCA claims.

This is a good thing. Many people are of the opinion that model weights don’t have copyright independent of the source code used to generate them. The source code and training data are all open. The model weights are a deterministic function of those. Claiming they have independent copyright is essentially the same as claiming copyright in a binary you compiled from open source code. Seems like a pretty good chance to get a court to agree with this.

There is a decent chance that model weights are not copyrightable (and I think that's the clearest definition of the law, I don't think they're a compiled work, at most they might be patentable. Particularly in situations where multiple models might converge towards the same weights, that seems more like a discovery than a creative act?

That being said, I don't feel like I know for certain how a court would rule. And I wouldn't like to fight Facebook about it in court regardless, that sounds like a pretty bad time even if the court sides with you. But on the other hand, Facebook might not be keen to test this either.

Any news on whether they're pursuing downstream weights? That's a really interesting question for me - if you started at the LLaMA weights and arrived somewhere else by supplying additional training data and computation, any infringement claim would be very, very murky. I also think it would be practically difficult to prosecute such a claim because it would necessarily involve comparative questions about the consumption of unlicensed training data to generate the original weights. It would certainly make for interesting law.

Carious timing given that the US copyright office itself does not consider these weights to be copyrightable per its statements just a few days ago. I hope someone takes Facebook to court.

People are arguing about whether model weights can be copyrighted, and who knows how courts will eventually rule. But if a site gets a DMCA takedown notice, they are taking a huge gamble if they don't comply: if courts do decide that Meta's copyright is valid, the monetary damages for an infringer could be huge. So I would expect the demands to be honored by anyone who can't afford to be wiped out in court.

And here I thought FB was going to try a more open approach than OpenAI. Silly me.

If someone put in more work to fine tune LLaMA, it may qualify for fair use.

BitTorrent is the answer for these cases. This is just an observation.

undefined

So, to me, there are three cases, and none of them give Facebook the standing for this to stick (but, of course, they can throw around a DMCA takedown and then require someone to file a counter-claim and maybe a lawsuit or whatever and that is going to be kind of pointless and likely quite expensive for sillysaurusx to do here; I think this takedown noticing law sucks in no small part because of that power differential).

1) You can't copyright weights. A lot of people believe this. I am not sure this is true. I think it might be that there is a fair use argument that the weights are transformative, but having fair use on your infringement doesn't imply a lack of the original copyright being owned by someone. But like, this might be true, and it is not an unreasonable stance.

2) The model weights are a derived work of the training data. This feels right to me, frankly, as much as it irks a lot of people on Hacker News who are excited to use Copilot (or owns shares of Microsoft ;P). In this case, Facebook does not own the copyright as they purposefully used an open training set from third parties (including Wikipedia and OpenCrawl) as a counter to OpenAI's proprietary one.

3) The model weights are a derived work of the training code, in the same way a binary is a derived work of its source code (vs. #2 where the code is a compiler and the source code is the training set). In this case, Facebook would own the copyright... but the resulting binary program--as distributed in the machine interpretable (and executable) format of the model weights--must be GPL as Facebook amazingly used that as the license for their training code.

The interpretation of events that would allow Facebook to have some hope of arguing that the model weights are their own work--that maybe the code is more of a tool like Photoshop and they have a fair use claim on the training data--would imply something that simply is not true: that they are adding some (hopefully extensive) form of expressive input above and beyond those two inputs, in the way someone using Photoshop does when they remix someone else's art into a transformative work.

However, Facebook definitely isn't doing that: they have provided absolutely no expressive input or intent on top of those two inputs, one of which they do not own and the other of which they chose to license for us under GPL! To make this a bit clearer, separate Facebook into two parties for a moment--one which developed the tooling and the other of which ran it--to determine which of the various parties you think owns this result: if you download a program someone else wrote and click a button to run it on some data someone else owns, you simply do not own the copyright on the result. (edit: I wrote some more on this argument in the following linked comment.)

https://news.ycombinator.com/item?id=35293068

The only thing I can come up with, if I try really really hard to steelman Facebook here, is: maybe, if I were to release a binary to the world that is a compiled copy of code that I also simultaneously released under GPL, the binary might technically have been compiled from an internal pre-licensed (but identical) copy of said code; and so, while if you compile the same binary from the licensed code you get an identical output that is a derived work with GPL rights, when I do that I don't... but this feels like a perilous argument to make as you are going to have such a hard time showing that this was a reasonable way to infer your intent with the simultaneous release.

(Of course, the person who originally agreed to the terms of service attached to the download they got is a totally different matter, but that doesn't mean there are none. There are a lot of limitations to what people can extract out of you if you violate a contract. So like, by having explicitly agreed to those terms with Facebook that person should not have given the world the weights--not because they are copyrighted but merely because they were secret--and there might be some kind of ramification... but, I do not believe that would possibly apply downstream to sillysaurusx.)

(Note: I am not a lawyer. I spend a ridiculous amount of my time vs. a normal engineer working with copyright and both hearing and making arguments about copyright, including directly to the Copyright Office at the Library of Congress as part of my work on Cydia and my efforts to push back on parts of the DMCA alongside lawyers from the EFF... but like, it would be foolish to read my comment here and then embark on a project to do something that might massively infringe on someone's copyrights without running it past a real lawyer. I, certainly, have actual lawyers.)

clone repos to your computer, dont just fork them

the fork disappears when the parent repository is taken down, clones stay on your machine

upload to IPFS. you can pin on IPFS for free with filecoin. Use filecoin nodes to pin on ipfs at https://web3.storage

Might be scared of more negative press, if the model generates something controversial ?

Why would I pay for LLAMA ???

I think it remains to be seen if weights can be copyrighted at all.

Insert Sun Tzu quote from the CEO here. This is war, after all.