Basically, they're claiming that the law says that any experiment on people, by anyone, requires informed consent. Every business with customers breaks this law. "Let's see if substituting medium roast espresso beans raises sales!" Sorry.

This isn't the law; it's a set of guidelines for federal agencies to follow. It'd be illegal if the department of health and human services or the FDA pulled this type of stunt, because they've voluntarily agreed not to do that by amending their rules and regulations to comply with the common rule that this blog post refers to (http://www.hhs.gov/ohrp/humansubjects/commonrule/)

Facebook and OKCupid are, at least not at the moment, part of the federal government.

"from the horrors of Nazi medicine to the deliberate withholding of care from syphilis victims in the Tuskegee experiment"

This is a deeply offensive comparison. Reading the complaint letter makes me wretch.

The letter that the authors submitted to Maryland's attorney general (http://james.grimmelmann.net/files/legal/facebook/MDAG.pdf) point out that Maryland's HB 917 -- which essentially applies the federal Common Rule to private companies who do business in Maryland -- defines "research" as "a systematic investigation ... designed to develop or contribute to generalizable knowledge."

That definition is so vague that practically any sort of interaction with a person in which any information about the interaction is recorded and analyzed can be construed as research. If a Girl Scout troop makes a graph of which neighborhoods buy the most cookies ... boom, that's research on human subjects.

I'd be interested to know whether any entity has been prosecuted for violating HB 917. I can't imagine that the extreme vagueness of the definition would hold up against a determined challenge.

What puzzles me is why James Grimmelman, J.D., and Leslie Meltzer Henry, J.D., think that the ultra-vague definition of "research" is legally tenable. Like, can't they anticipate the objections to the definition -- that it effectively bans all sorts of everyday actions that no reasonable person wants to subject to these sort of legal hurdles?

"This is why we cannot have nice things."

OKCupid's blog is super interesting but they might have to discontinue it if it is going to generate legal challenges like this one. The only reason we know about their A/B testing is because they openly told us about it, which if they get prosecuted for it might be a mistake.

I kind of liken it to the "never apologise" rule many companies now have. It is when common sense/public interest run into cold legal restrictions.

It seems like this "law" as interpreted the authors to ban A/B testing would violate the 1st Amendment rights of companies to uncensored speech with their customers. I hope that they challenge FB in court so it can be overturned.

A/B testing is also illegal by this standard.

This makes no sense to me. The algorithms employed by Facebook, OkCupid et al. are not published and presumably changed regularly both to better serve their users and to advance commercial imperatives - apparently this is OK but if you measure the outcomes and call it an experiment it's not?

I don't buy this, as long as all representations made to customers about the service are met, it seems perfectly reasonable to tweak the already unknown algorithms behind the scenes for whatever reason.

The Maryland House Bill 917 is at http://mlis.state.md.us/2002rs/billfile/hb0917.htm . The gist of it is:

(a) Compliance with federal regulations. -- A person may not conduct research using a human subject unless the person conducts the research in accordance with the federal regulations on the protection of human subjects.

(b) Scope to include all research. -- Notwithstanding any provision in the federal regulations on the protection of human subjects that limits the applicability of the federal regulations to certain research, subsection (a) of this section applies to all research using a human subject.

My problem is, I'm not sure what MD 917 (b) means. I'll start with §46.102 (a) (1) at http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html#... :

> (a) (1) Research that is conducted or supported by a federal department or agency, whether or not it is regulated as defined in §46.102, must comply with all sections of this policy.

Is MD 917 (b) supposed to mean that §46.102 (a) (1) applies to all human research in Maryland? I believe that's the case, because §46.102 (a) (2) only applies to those organizations which are regulated by a federal department, so would already be covered under the law.

Section §46.102 (b) has a list of human research which is exempt. The most relevant for Facebook and OkCupid appears to be:

> (2) Research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior, unless: (i) information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and (ii) any disclosure of the human subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or reputation. [Note: does not apply to children.]

If A/B testing falls under 'educational tests' or 'survey procedures' then it's exempt.

Otherwise - and this is another tricky part - §46.102 (c) says "Department or agency heads retain final judgment as to whether a particular activity is covered by this policy."

Who is the department head for Facebook and OkCupid? §46.102 (a) defines it as:

> (a) Department or agency head means the head of any federal department or agency and any other officer or employee of any department or agency to whom authority has been delegated.

MD 917 doesn't have the information for how to interpret §46.102 outside of the federal context for which it was written.

An alternative interpretation of MD 917 (b) is that no human testing is exempt, including §46.102 (b) (2) which allows surveys. Instead, all testing must go through an IRB.

However, if that's the case then the other exceptions of §46.102 (b) are also no longer allowed, including

> (4) Research involving the collection or study of existing data, documents, records, pathological specimens, or diagnostic specimens, if these sources are publicly available or if the information is recorded by the investigator in such a manner that subjects cannot be identified, directly or through identifiers linked to the subjects.

which would require IRB review for every historian working out of previously published materials.

Therefore, I don't think this alternative interpretation is correct.

Which means I don't know if MD 917 really applies to OkCupid or Facebook. (The Facebook research was published in a journal which follows the Common Rule compliance; I presume the journals have a way to resolve the question of what "department head" means when there is no department head - it must have come up before. I only investigated MD 917 compliance here.)

_EDIT_ : The MD attorney general clarified this at http://www.oag.state.md.us/Healthpol/hb917letter.pdf .

> Implicit in the federal regulations, and therefore in House Bill 917's mandate for compliance with these regulations, is the requirement that a person have in place a reasonable process for determining whether research falls within one of the exempt categories.

So determining if A/B testing is exempt comes down to if there's a reasonable process for determining if it's exempt. It's not clearly illegal.

So.... are A/B testings now banned?

Maybe they were illegal. The point is they are companies and can do whatever they want, largely without scrutiny. I mean this in the sense that it is difficult to bring charges against them, as evidenced by basically the entire financial sector.

When things like this are published it just serves to dissuade them from PUBLISHING the results of the experiments but not from actually CONDUCTING THEM.

I would much rather know when they do things like this than have them continue to conduct them in secret.

I encourage people to read Section 46.102 [1] (page 4) of the federal regulations that Maryland is enforcing. It lays out the definitions of all these terms (as a mathematician, I put the highest weight on definitions), and in particular section (i) on "minimal risk" is clearly in Facebook/OKC's favor.

[1]: http://www.hhs.gov/ohrp/policy/ohrpregulations.pdf

The Common Rule AFAIK only applies to places that get government funding, so I don't see how this applies to research FB does on its own.

Here

https://medium.com/@JamesGrimmelmann/illegal-unethical-and-m...

the author explains more clearly his position:

"Federal law — primarily the so-called “Common Rule”— ( http://www.hhs.gov/ohrp/humansubjects/commonrule/ ) regulates research on people in the United States. The details are complicated, the gist simple. If you engage in “research involving human subjects,” you must have two pieces of paper before you start. You need a signed informed consent form from the person you’re experimenting on, and you need approval from an IRB (short for “institutional review board”)."

The phrase "generalizable knowledge" seems crucial. I am not a lawyer, and have not done human subjects research, so hope someone else can comment.

However, looking around, it sure sounds like this would not normally cover A/B testing, a business changing its prices and tracking results, or similarly "local" investigations.

Here is one IRB's interpretation of that term: http://www.irb.wsu.edu/definitions.asp. Others seemed similar.

The typical TOS these days turns users into corporate serfs ... I'll bet they respond to these complaints by alleging the users had granted consent.

Would it be correct then that any change in layout or diversity in Maryland newspapers, periodicals, and publications would also be illegal? For instance, asking people they prefer a Sports section prior to Entertainment, or having Politics or Local Interest above the fold? Would it be ok to make the changes given that the effects were never measured?

OP's position is extreme, and I feel like any negative publicity to Facebook or OkCupid over this is going to ultimately have the effect of punishing them for saying that they ran these experiments.

Changing the way content is ranked by your private service is not in the same ethical territory as the Tuskegee Syphilis Study. It's just not.

That said, there is an interesting discussion here, which is: where is the line? Netflix runs an experiment (and whether it chooses to collect data) every time it changes its ranking algorithm, but I think we'd agree that it's ethically OK in that case. What about those sleazy mobile game companies that use data science and heavy experimentation to find and target "whales" who have an in-app-purchase addiction problem? Not ethically OK. I don't know where the line is, but it's clear to me that some tech companies are on one side and some are on the other.

We changed the url from http://laboratorium.net/archive/2014/09/23/facebook_and_okcu..., which points to this longer piece.