There are a lot of lessons to be learned from Diaspora, beyond the (brilliant) points made about security.

1. When media hype provides you with $200k, you're still best served by bringing people's expectations down to earth. There was no way they were going to be able to build anything approaching a Facebook killer in 3 months, and it would have been best if they would have made that clear in the beginning.

2. There are a number of projects like Diaspora that have been working for years towards the same exact goal. The only way Diaspora could have succeeded where those have failed (or not-yet-succeeded), is if they had properly articulated where they would go right where others had gone wrong. If you can't do that, you probably don't have the perspective necessary to take on such a huge project.

3. We all need to be less susceptible to the story of the "boy wonders" taking on the establishment. Between Diaspora and Haystack+, these should be sobering lessons about the dangers of hype and a good human-interest story when the result we need is stable, well-written code.

+ http://blog.jgc.org/2010/09/myth-of-boy-wizard.html

4. Open source isn't magic. Rails isn't magic. Having a good idea and a whole lot of heart isn't magic. It's true that the software world isn't exactly a meritocracy, but at the same time, we need to recognize that you have to be able to build something generally usable, and if you can't, there's nothing that will save you except for harder work and learned lessons. Some bugs will be fixed by the interest in Diaspora, but there are big architecture questions here that need to be resolved, and coordinating that democratically through the internet in a sea of strangers is a logistical nightmare. And while rails does provide a lot of functionality out-of-the-box, a project of this size isn't held up by how long it takes to write the photo-uploading code, it's held up by the big-picture stuff that rails can't really help you with. In the end, programming is programming. You need specs, mockups, user stories, documentation, all kinds of unsexy stuff.

I think we'll probably see Diaspora stabilize into something usable at some point. But I'm very doubtful that will be anytime soon, and I'm especially doubtful that it will be before the other projects (Elgg, Appleseed, OneSocialWeb, StatusNet, etc) mature into the facebook killer people want to see.

Building the kind of open source social networking software necessary to take on Facebook at it's own game is such a massive, complex undertaking, and it's such undiscovered territory, that there really is a big disadvantage to being the new kid on the block.

"For example, if you were logged in to a Diaspora seed and knew the ID of any photo on the server, changing the URL of any destroy action from the ID of a photo you own to an ID of any other photo would let you delete that second photo."

When I was working as a pen tester I would completely scold developers for letting this happen - telling them that with everything we know today about security and good programming practices there is no way you should allow that to happen. Off by-one bugs, timing attacks etc. are more excusable, but this, this is just amateur hour.

That was 11 years ago.

I can probably go all night on this, but a couple things from a quick read of this (very good) post:

First, mass assignment.

The answer to mass-assignment bugs is "attr_accessible". Accessible attributes can be set via update/build/new; nothing else can. Every Rails AR model should have an "attr_accessible" line in it.

I've met smart dev teams working under the misconception that attr_accessible means "these are the attributes that can be changed based on user requests", and so virtually everything is made accessible. No! If something's not attr_accessible, you just set it manually (user.foo = params[:user][:foo]). It's not painful and the extra line expresses something important ("this is a sensitive attribute"). Attributes are inaccessible until they prove themselves mass-assignment-worthy.

Second, the string interpolation in the regex.

Real quick: don't ever let users interpolate arbitrary strings into regular expressions. Regular expression libraries are terribly complicated and not very well tested. To illustrate (but not fully explain) the danger here, run this line of code:

     ruby -e "'=XX===============================' =~
     /X(.+)+X/"

Oh, one more thing: I appreciate Patrick's take on systems failures breaking Rails apps before underlying crypto flaws will, but even if they had protected their keys, their crypto wouldn't have worked. Don't build things that require crypto. You aren't going to get it right.

Lesson learned: Never let the outside world see your First Big Project Ever.

This is what Fred Brooks would have called the First System. Everybody builds this thing at the beginning of their career, and it's always this embarrassing. Mine, in 1996, took this a step further and actually prepared SQL statements in javascript before submitting them to the server to run. Yours probably did something equally bad. It's the rule of the First System. It's where we learn all the lessons we need so that we can do things right the Third time (I'll spare the Second System description for the time being...)

The key though is to make sure that nobody ever sees that code. Hopefully it will be locked away in some intranet vacation-time-planning app that nobody will ever dig into. That way you can look back at it in shame, but few others will ever know about it.

So here we have a team of people who have clearly never built anything at all, trying to learn on the job while being scrutinized by the entire world and actually submitting their code for public review.

God help them.

I'll admit, when people came down hard on them vis a vis security, I figured it was just a bit sloppy as a preview release.

I was wrong.. these aren't really security "holes" as that's not strong enough a word. I think the best way to put it is they accidentally created the first social network wiki.

It's not that the pre-alpha Diaspora has insecurities that bothers me, it's the whole execution.

What I really would like to see is a documented protocol - based on XMPP or some other established, well-tested protocol would be good, but if not then at least something.

Once you have that protocol - which tells you how Diaspora "seeds" communicate securely - you can let others build their own implementation, using Rails, PHP, Python, doesn't matter. Sure, release a reference implementation in Rails, but the protocol is the most important thing.

Unfortunately what we have is just another Facebook clone done in Rails, which is disappointing.

"NoSQL Doesn’t Mean No SQL Injection"

I lol'd. Mind if I use that?

MongoDB is harder to secure and filter because you have all of Javascript to worry about, rather than just SQL (and where most servers can escape arguments themselves through prepared statements etc.).

SQL databases are also well understood (for eg. in MS-SQL I can stop the remainder of the statement from executing with '--'). MongoDB with its JS engine is still a big unknown.

One more bug in the first code snippet is their use of find_by_id means that @album could be nil, causing an error when they attempt to edit it. Most people would use @album.find and then catch the error/show a 404.

I realize these guys are in college, but they really should have (a) brought people's expectations in line with their abilities and (b) reached out to experienced developers to help them out. Intridea probably would have given them a few developer hours per week to help code, advise them, and so on. Just a little input and guidance would have saved them a lot of grief.

The biggest problem I think, and I speak from experience, is that some of the best rails tutorials leave out a lot of important security issues entirely. Some of them might do a trivial paragraph on password encryption, but most of them leave out any mention of these kind of basic security flaws.

Even if the writers didn't explicitly write chapters dedicated to security, obviously even that wouldn't be enough, they should at least note which code is entirely unsafe in production, and where you can learn more about it.

Every rails book is filled with this sort of code that people learn, and then use.

This is a great post and shows how awesome of a community we have. The Diaspora guys are a bit behind when it comes to solid programming, but Patrick (among others) reviewed the code, noted flaws, reported and advised them on how to fix them, then wrote a great post explaining what was wrong to help others avoid the same mistakes. Stuff like this is both educational and helps the programming community move forward. Thanks to Patrick.

Thanks for the web-app security pointers in the post: the RoR security guide[1] and the OWASP Top 10[2]. These should be required reading for anyone making a public web application.

[1] http://guides.rubyonrails.org/security.html (Well-written, like the other guides. Totally worth reading fully).

[2] http://www.owasp.org/index.php/Top_10_2010-Main (Open Web Application Security Project's top application security risks for 2010)

Not to be totally nitpicky but if they're using any recent version of Rails (I haven't looked at the source yet), the DESTROY action doesn't respond to GET by default. That doesn't change the fact that they don't scope deletes to the logged-in user's assets.

Would appreciate if more articles like this are posted on HN, useful and practical!

I knew Diaspora had problems when I got an email from the guy who hosts Openspora telling me that [two] other people had the same username.

I like that the phrase "Lil Bobby Tables" to reference dropping the database via SQL injection has entered the hacker vernacular.

What would be a nice one-page security guide would be a 'lil bobby tables' guide to databases - SQL injection for any database - (SQL or NoSQL) - the goal being to help developers prevent these attacks.

Did anyone really believe that a bunch of guys were going to write a secure privacy-based facebook-killer application in three or four months?

That's a challenging proposition even for experienced teams.

Though we all know that they warned us that the software was still full of bugs and treated as experimental.

Patrick says that doesn't matter, that they haven't got the foundation right and anything built on top of it will likely fall.

Maybe they can still iterate and fix those things. Maybe they will scrap large sections of the code and re-write it properly. I suggest they keep trying and learn something from all of this. Us older geeks can be pretty harsh sometimes. We often expect new-comers to not make the same mistakes we did once. That's how we learn though, so don't take it the wrong way.

Next time just don't promise more than you can deliver.

Wow, those are some pretty amateur mistakes. Just taking whatever data you got from the user with no checks? Really? And that's not something you fix later. That should stick out like a sore thumb the moment you write it, making you check that user == logged_in_user before you even move on. Wow.

I appreciate the two links with how to handling security issues in web application.

As many have pointed out, they do not teach this stuff in University and acquiring such knowledge you tend to have to be very proactive about your development if you do not have industry experience.

So thanks a lot for sending these nuggets our way.

You stated that the team is manifestly out of their depth in terms of web security, how do you suggest that they proceed given their month deadline? Can they pay a security expert to resolve these issue or no one wants to come within a mile of this?

I had my doubts about Diaspora, but now I know for sure: it will be a complete fiasco. Those are horrendous errors that show that those guys have completely no idea about web programming - and I can't see them learning it quickly (certainly not before planned release date of the final version). Sad thing.

What idiots those Diaspora guys are! All that excellent security consulting, for free! They don't know the first thing about software development! It makes me so mad, I'm going to write up a carefully researched and detailed account of other errors they've made! Then they'll see how clueless they are - again! Ha, what amateurs!

Excellent article. I can see why under the kind of time pressure they have been under there would be issues like these.

Interesting I was thinking in comparison to PHP where a beginner would much more likely be individually assigning each input into a SQL statement. Sure there are all kind of possibilities that can go wrong there to but mass assignment looks like it makes it so easy for someone to shoot themselves in the foot.

I've always wondered about this (not being a code-monkey-ninja-wizard, myself)...

If you open source something, unless it's perfectly written, wouldn't the hacking potential be... near 100%? If everyone can see how you do everything it seems like even a minor slip up will potentially surrender your site.

Could someone explain this (I'm probably missing a piece of the puzzle I can't place)?

It is funny, I have looked at the code and am not a big ruby/rails person, but see obvious flaws and security holes. However, a lot of the discussion around these have been either in the "it is amateurish" bucket or the "are they expecting help from the OSS community? that is leveraging eyeballs, that may not come".

Now, while it hasn't been their primary intension (most likely), to have the OSS community (and others curious about the code) provide fixes and feed back, they sure have gotten a lot of input and advice. I also suspect they have learned quite a lot through the feedback - positive, constructive, and inflammatory. You really can't ask much more -- aside from some direct help.

While I have no plan to run my own Diaspora node, I do look forward to seeing how the code and project evolve.

undefined

After reading the article, also read this comment: http://www.kalzumeus.com/2010/09/22/security-lessons-learned...

The comment negates some of the statements made in the post

Well, I for one am glad that I learned from articles like this. It's a hell of a lot easier to read discussion from "real programmers" and learn from them. Now that I'm in college stuff like this is second nature.

undefined

OK, you've made headlines. Now, where are your patches? ^_^

I don't think this is that big of a deal. Pretty much every developer I know has learned about security through this exact process. Either a senior developer or user exposes the flaw and smart, but new, developers quickly realize the didn't understand the attack angles. Without concrete experience it's pretty hard to appreciate how exactly these attacks work. But after a few exploits you start getting paranoid, understand the basic problems, and are always thinking and reading about how your code might be missing something. These guys seem reasonably smart so I expect after a few months of exposure, they'll be fine.